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THE PURPOSE OF THE STUDY WAS TO REVIEW AND SYNTHESIZE 
THE RESULTS OF RESEARCH ON JOB PERFORMANCE AIDS AND EXPLORE 
THEIR CURRENT AND POTENTIAL ON INDUSTRIAL MANPOWER 
UTILIZATION. JOB PERFORMANCE AIDS ARE AUXILIARY INFORMATION 
STORAGE DEVICES WHICH PRESENT JOB INSTRUCTIONS OR REFERENCE 
DATA FOR USE BY THE EMPLOYEE DURING THE COURSE OF HIS JOB 
PERFORMANCE. AN UNDERLYING ASSUMPTION WAS THAT JOB 
PERFORMANCE AIDS COULD BE VIEWED AS A MAJOR TOOL FOR MATCHING 
JOB REQUIREMENTS WITH AVAILABLE MANPOWER RESOURCES. FINDINGS 
WERE DERIVED FROM A REVIEW OF THE LITERATURE AND FROM A FIELD 
STUDY OF PERFORMANCE AID UTILIZATION IN 12 ELECTRONICS 
ASSEMBLY ORGANIZATIONS AND PATIENT CARE AREAS OF 12 
HOSPITALS. THE RESULTS SHOWED THAT DEPENDING UPON THEIR 
DESIGN AND USAGE, JOB PERFORMANCE AIDS COULD COMPENSATE FOR 
LACK OF TRAINING AND EXPERIENCE, IMPROVE JOB QUALITY, AND 
INCREASE PRODUCTIVITY. THE IMPACT OF JOB AIDS WAS 
PARTICULARLY APPARENT IN COMPLEX OR LENGTHY TASKS, 

DIMINISHING IN EFFECTIVENESS AS TASKS WERE SUBDIVIDED INTO 
SHORT CYCLE, REPETITIVE OPERATIONS. A BASIC FRAMEWORK FOR 
ADVANCING THE FIELD OF JOB PERFORMANCE AIDS WAS PROPOSED, AND 
A SERIES OF RESEARCH QUESTIONS WAS LISTED. IT WAS RECOMMENDED 
THAT THE DEPARTMENT OF LABOR CONSIDER THE ROLE IT MIGHT FLAY 
IN OVERCOMING THE LACK OF COMMUNICATIONS WHICH UP TO NOW HAS 
HINDERED THE DEVELOPMENT OF THE JOB PERFORMANCE AID FIELD. 
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PREFACE 
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Labor, Office of Manpower Policy, Evaluation and Research and the Philco-Ford 
Corporation Western Development Laboratories. The study was conducted in the 
Personnel Subsystems Department, Mr. T. W. Reiland, Manager. Miss Catherine 
Christgau served as contract monitor for the Department of Labor. 

The authors are grateful to the many individuals and organizations that supplied in- 
formation so generously during this exploratory study. Particular appreciation is 
extended to the Palo Alto-Stanford Hospital Center; in particular to Mr. Samuel 
Virts, Assistant Director, Employee Relations -Communications and Training; to 
Mrs. G. Darlene Lutz, R. N. , Director, Nursing Service; and to Miss Marjorie 
Keys, R. N. , Supervisor of In-Service Education and her staff. 

Among the Philco-Ford personnel who contributed to this report, special recogni- 
tion is due Dr. R. A. Goldbeck for his review of the draft report and his help in the 
preparation of the annotated bibliography. Mr. James Casalegno was also of major 
assistance in developing the bibliography and taking care of the many details involved 
in producing the final report. Mr. Wilbert Cox participated in the early study pre- 
parations and provided helpful comments on the final manuscript. 
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SUMMARY 

The present investigation was designed to review and synthesize the results 
of job performance aid research and application and to explore the current and po- 
tential impact of such aids upon industrial manpower utilization. The study involved 
a review and analysis of relevant research literature and reports of job aid applica- 
tions, summarie,'} of which are included in an appendix. In addition, a field study 
was conducted of job performance aid usage in two distinctly different occupational 
areas: electronics assembly and patient care. Interviews were conducted in 24 
organizations, twelve representing each occupational area. 

The results show that, depending upon their design and usage, job perform- 
ance aids can compensate for lack of training and experience, improve job quality, 
and increase productivity. Evidence is generally lacking concerning the extent to 
which job performance aids can compensate for basic aptitude deficiencies. The 
impact of job aids is particularly apparent in complex or lengthy tasks, diminishing 
in effectiveness as tasks are subdivided into short cycle, repetitive operations. 

A basic framework for advancing the field of job performance aids is pro- 
posed and discussed. Recommendations are made for comprehensive job aid de- 
velopment and demonstration efforts. On a. more basic level, a series of research 
questions are posed, the solutions to which would contribute to better understanding 
and long-term utilization of job aids. 

Finally, it is recommended that the Department of Labor consider the role it 
might play in overcoming the significant lack of com.munications v/hich up to now 
has apparently hindered the development of the job performance aid field. Specific 
suggestions are inclu|iied for fostering improved communications in the job aid field. 
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INTRODUCTION 



One of the major goals of the research sponsored under the Manpower Develop- 
ment and Training Act is to contribute to the development of new perspectives and 
solutions to manpower problems. As a result, a wide variety of research has been 
conducted on many of the specific problems associated with the impact of technolog- 
ical changes. Solutions are being sought to the employment problems of the nonwhite, 
the older worker, the handicapped, and other disadvantaged groups. At the core of 
many of our manpower problems is the very basic question of how can we improve 
the effectiveness and the efficiency of our methods for achieving satisfactory human 
performance in a w^' rk environment with a rapidly accelerating rate of change. The 
wide-scale attack upon educational and training problemr testifies to the vital con- 
cern for achieving a better balance between human performance capabilities on the 
one hand and changing job demands on the other. 

The present study is concerned very directly with the improvement of methods 
for facilitating human performance; but its focus is not upon the area generally de- 
fined as training. Rather, a primary objective of the study is to define the field of 
job performance aids as an element in an overall human performance system and to 
explore the potential of the field for improving our efforts in achieving adequate job 
performance. 



While the nature of job performance aids will be explored in depth throughout 
the report, they can be defined here as auxiliary information storage devices which 
present job instructions or reference data for use by the employee during the course 
of his job performance. For purposes of this discussion, the particular form and 
construction of the performance aid is not important. Whether it be a checklist, a 
nomograph, or a computerized job information system, the main consideration is 
that the "device" assist the worker by storing or otherwise handling information . 
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Accordingly, other types of equipment used on the job; e. g. , tools, jigs and fix- 
tures, etc. , are not included in our definition of job performance aids. Job per- 
formance aids are designed specifically to facilitate job performance by relieving 
certain job demands; for example, the demand for memorizing complex, lengthy, 
or infrequently used job sequences or supporting reference formulas, values, etc. 

This report describes an exploratory study of job performance aids and their 
current and potential impact upon manpower selection, training, and utilization 
practices. Specifically, the research method involved a comprehensive review and 
analysis of research and development reports, supplemented by a field study of per- 
formance aid utilization in two selected occupational areas: electronics assembly 
and patient care. The following section describes the basic concept of job perform- 
ance aids. Research and application literature covering both military and industrial 
job aid experience is then examined. The next two sections describe the procedures 
used and the results obtained from the electronics assembly and patient care sur- 
veys. In order to provide a foundation for further study, possible bases for cate- 
gorizing job performance aids are proposed and relationships between such aids 
and the training area are discussed. Tentative conclusions are then drawn concern- 
ing the impact of performance aids upon training, selection, job quality, and pro- 
ductivity. The resulting implications for routine tasks and for technical work areas 
are discussed. Recommendations for research and development are provided and 
policy implications are then examined. Finally, to provide a foundation of reference 
information and to enable the reader to identify particular reports worthy of read- 
ing, an annotated bibliography of publications relevant to the job performance aid 
field is provided. 
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' THE CONCEPT OF JOB PERFORMANCE AIDS 

j One of the most common examples of job performance aids is the instruction 

sheet that accompanies an unassembled toy or piece of furniture. Properly develop- 
j ed, this little sheet of paper can make the difference between virtual chaos and 

achieving a rather complex new feat (e. g. , bicycle assembly) without any special 
j training and with only a minimum of previously acquired skills, such as reading 

and simple tool handling. And yet as frequently as we have been confronted with 
j specific illustrations of good and, more often, inadequate performance aids there 

^ has been pitifully little effort devoted toward exploiting the area of external inform- 

- ation storage and presentation as an integral part of human work performance 

) development. 

j Rather than forming a conscious part of job design, most of the efforts to pro- 

vide information support appear to have been approached as an afterthought. Con- 
1 sequently, job aids are more often remembered for their frustrating inadequacy 

instead of for their contribution to fostering human performance. Their potential 
i for influencing such decisions as what level of personnel are required for job per- 

* formance and how much training must be provided before an individual can function 

J adequately on a job has largely been ignored in the industrial work area. 

j'i 

i 

It is not meant to imply that the field of performance aids has been completely 
disregarded in the industrial work area. There are certainly many examples of 
effective job aids in use today. The lubrication chart in the filling station, by 
specifying the activities to be followed and showing the location of the parts to be 
lubricated for each model and year of car, is an excellent example of a type of job 
aid which substitutes for the lengthy training and experience that would otherwise be 
required if the mechanic had to rely exclusively upon his memory to accomplish 
such tasks. With only a minimum of training, any gas station attendant utilizing the 
job performance aid can lubricate most vehicles to the same level of performance, 
and very likely with greater reliability, than would be achieved by a skilled mech- 
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anic who had to rely entirely upon his previous training and past experience. 

Despite such examples, job performance aids as a special field have been vir- 
tually overlooked. This omission can be detected in our efforts to solve the press- 
ing manpower problems cited in the previous section. Whether or not expressly 
stated, the assumption that imderlies most of our efforts to combat poverty and un- 
employment is that, given a particular set of job requirements, there are only two 
vehicles for obtaining human work performance - selection and training. This 
assumption is illustrated very simply in Figure 1. If we extend this concept to its 
logical conclusion, it can be seen that to the extent individuals enter the job mar- 
ket lacking in specific job knowledges we have no alternative but to lengthen the 
training period or in other ways expand the scope of training as illustrated in the 
figure. 

The almost total commitment to training as the solution to the problems of the 
disadvantaged can be seen underlying practically every major project associated 
with the Poverty Program. Without in any way detracting from the legitimate em- 
phasis upon training and the need to expand and improve our training efforts, the 
hypothesis underlying the present study is that training is not the only vehicle for 
aligning job requirements with manpower resources. The assumption that job 
performance aids can be viewed as a major tool for obtaining humap performance 
provided the major stimulus for the work described in this report. 

Adding another element to performance development can provide greater flex- 
ibility and broadened perspective toward the solution of manpower problems. Effec- 
tive job performance aids, by reducing the information retention load in a job, can 
ideally serve as an alternative to the continual expansion of training that must 
otherwise result as jobs become more complex or as the input quality of the work 
force decreases. The potential effect of job performance aids upon selection and 
training is illustrated in the model shown in Figure 2. 
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HYPOTHETICAL ATTAINMENT OF JOB PERFORMANCE REQUIREMENTS 
BASED ON SELECTION AND TRAINING ONLY 
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SATISFACTION OF JOB PERFORMANCE REQUIREMENTS 
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From this model, one can envision that effective job performance aids can be 
expected to result in; 

• Reduction in training time 

• Lowering of selection standards 

Other potential benefits include: 

• Improved quality and reduction in job errors 

• Increased reliability of job performance 

• Greater productivity 

• Increased scope and complexity of tasks performed 
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It must be stressed that the purpose of the foregoing discussion was to provide 
an initial structuring of the field of job performance aids and to cite some of the 
contributions which conceivably can be gained from such an approach. While at 
this stage the concepts and contributions of job aids are largely theoretical, they 
are not without some basis in empirical research and practical application. It is 
the goal of this report to attempt to bring together the relevant research and some 
of the more systematic applications of job performance aids and from this to pro- 
vide a basis for a better assessment of the field as a tool for improved manpower 
utilization. 

COMPARISON OF TRAINING AIDS AND JOB PERFORMANCE AIDS 



I 






As indicated earlier, job performance aids are defined as documentation or 
auxiliary information storage and presentation devices used on the job to provide 
instructions and reference information. They are designed to facilitate employee 
performance through direct usage either during the course of task performance or 
perhaps immediately prior to beginning a task. A handbook of automobile engine 
specifications, for example, is a job performance aid. Its prime function is to 
serve as a reference guide for mechanics engaged in engine adjustment and repair. 
While students should be taught how to use the aid efficiently, one would not normal- 
ly use such a handbook in the classroom for the purpose of teaching students the 
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specifications of all engines^ On the other hand, a mock-up or other display show- 
ing the flow of gasoline in an automobile engine is clearly a training aid and is not 
intended for use by mechanics on the job during engine repair. 

While indeed individuals may learn certain things during their use of job aids; 
the major function of such aids is to support job performance and not to achieve 
long term retention. In contrast, classroom training aids, some of which may 
nevertheless be appropriate for on-the-job use, have the major goal of fostering 
the learning of a task. Of course, it is obvious that job performance aids can fre- 
quently be used to advantage in the classroom to develop skills and foster attitudes 
conducive to on-the-job usage. Table 1 presents a rough comparison of the primary 
functions and key features of training aids as compared with job performance aids. 
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REVIEW & ANALYSIS OF JOB PERFORMANCE AID 
RESEARCH & APPLICATION 



The first phase of the study, which was continued up until the drafting of the 
report, involved an intensive review and critical analysis of the reported research 
and application of job aids. Major reference indexes were reviewed both in the 
scientific and technical area and in the business and general publication area as 
well. The specific sources of reference information are listed in Appendix B. 
Articles that appeared relevant to the subject area were identified and were obtain- 
ed for review and analysis. Despite the fact that the relevant literature is traceable 
back nearly 15 years, much of it consisting of governmental reports, in only a few 
instances was it necessary to rely upon secondary sources of information. 



The literature surveyed can be categorized roughly into two areas: 

(a) military research and application and (b) civilian applications. By far the 
greatest amount of applicable research was sponsored by the military, primarily 
the Air Force. With but a few exceptions, the articles in the civilian publications 
dealt primarily with reports of the development and utilization of specific perform- 
ance aids. Unfortunately, as will be discussed in depth later, the claims made in 
the trade publications for the efficacy of various types of job aids were generally 
unsubstantiated and accompanied by very little supporting data upon which to base 
an evaluation. 



Two other types of information served as an input to the research and applica- 
tion review phase. To ensure that the latest findings from recently completed re- 
search and on-going studies were reflected in the report, contacts were established 
and information obtained directly from investigators currently working in the job 
performance aid field or related areas. A list of the individuals and organizations 
contacted is provided in Appendix C. 
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In addition, reports and other descriptive material were accumulated on a var- 
iety of job performance aid equipment and other media currently in use or e^qpected 
to be available for use shortly. Where it appeared worthwhile, personal contacts 
were established with representatives of the manufacturers and distributors of 
equipment applicable or potentially applicable for use as job aids. In addition to 
providing the latest hardware information, these contacts also served as a source 
of information for the field study. 

AIR FORCE RESEARCH AND APPLICATION 

By far the major contribution to job performance aid research has. been made 
by the Air Force. More than any other organization, the Air Force and its support- 
ing contractors have identified and defined the job performance aid field as an im- 
portant element in system effectiveness. Understandably, the research has been 
directed toward the problems associated with complex weapon systems. 






The first of a lengthy list of reports on the conduct of research and develop- i 

1 1 

ment in this area appeared in the mid 1950's with a report by Berkshire (5) de- 
scribing the field evaluation of an aid for troubleshooting a portion of a radar set. 

From 1954 to 1958 the Air Force Personnel and Training Research Center (AFPTRC) 
sponsored a sizable number of studies most of which, like Berkshire's, dealt with ^ 

the development of sample job aids for particular systems or subsystems. The 

i 

primary emphasis in the research was upon simplification of troubleshooting, in- 
cluding research on what information should be displayed and on determining op- ? 

timum strategies for troubleshooting. Typical of this period is the work of Warren 
et al, in developing and evaluating a symptom-malfunction index as a troubleshoot- j 

ing aid for complex electronic systems (53). . | 

I 

1 



Numbers in the parenthesis direct the reader to the specific publication 
indexed and abstracted in Appendix A. 
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During this period, the Maintenance Laboratory of AFPTRC developed a major 
research program to explore the possibilities of materially decreasing technical 
training and achieving important increases in reliability of technician performance 
through improved methods for communicating technical information in operational 
situations. The program was planned to include the development and test of experi- 
mental job aid materials and devices as well as research on effective presentation 
techniques for use in printed guides and in film aids and other devices. Unfortu- 
nately, what was to have been a three to four year research and development pro- 
gram had to be terminated in less than two years because of curtailments in man- 
power and funds. Hence, many of the various aspects of the research could not be 
carried through to fruition at that time (24). 

The Air Fore*., interest in job performance aids was again expressed during the 
early 1960’s with the publication of a series of reports by Folley et al (17, 18, 19, 

20) under contract to the Behavioral Sciences Laboratory of the Air Force Systems 
Command. These reports reviewed previous military experience in performance 
aids, identified research problem areas, and described the development and tryout 
of a preliminary procedure for systematic job aid design. The Behavioral Sciences 
Laboratory continues to sponsor research directed toward achieving a better under- f- 
standing of the many variables which influence the effectiveness of performance 
aids in maintaining electronic equipment. Among the variables being studied are 
level of information detail, information format, data flow relationships, and per- 
sonnel aptitude (16, 50). 



o 

uc 



Two other research and development programs of the Air Force have definite 
implications for job aids. Currently underway is a project under the sponsorship 
of the Air Force Systems Command entitled Presentation of Information for Main- 
tenance and Operations (PIMO) directed toward improving system maintenance by 
providing a better system for presenting maintenance information to the technician. 
Specifically, the program is concerned with identifying what information is actually 
required by maintenance personnel, determing the best way of compiling such in- 
formation, and optimizing the means of presentation of information to the user. 



13 



PHILCO 




WDL Division 



PHILCO -FORO CORPORATION 















WDL-TR3276 

Specifications for the development of an audiovisual presentation device have re- 
sulted, based upon a feasibility study using a helicopter as the test system (37, 
p. 128-137). 

AlIso related to the area of job performance aids is the program of research 
under the direction of the Decision Sciences Laboratory of the Air Force Electronics 
Division (6, 7, 15, 44, 45). The major objective of these experiments is to develop 
design principles for automated training subsystems which could be incorporated 
into the design of future information bystems. 

Reported Effects of Performance Aids in the Air Force Environment 

The research effort outlined in the previous section covered a wide range of 
problems and encompassed studies dealing with the effectiveness of improved job 
performance aids as well as research into various factors involved in the design of 
these aids. This section will attempt to summarize briefly the reported effects of 
introducing performance aids into specific Air Force maintenance situations. 

Berkshire 's study (5) involved the development and preliminary evaluation of 
a set of troubleshooting materials for a portion of a radar set. These materials 
were intended to be explicit and detailed enough such that the mechanic would not 
need a complete understanding of the functional relationships among the parts of the 
equipment. Twenty-four trained mechanics participated in the preliminary field 
evaluation of the materials. Eighteen of the men had a range of experience from 
seven to thirty-one months with a mean of twenty-one months; while six of the men 
had little or no maintenance experience. Six different malfunctions were studied 
and each mechanic had the task of locating two malfunctions. The technician used 
his own methods for locating one malfunction and for the other employed the spe- 
cially designed troubleshooting materials. 

The experienced mechanics using their own methods frequently lost time through 
removal of the wrong components. The troubleshooting materials were estimated 
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to save on the average of two hours per malfunction (with time scores corrected to 
include component replacement and checking time) when used by trained and ex- 
perienced mechanics. Use of the written instructions, called trouble locators, 
greatly reduced the errors made by both experienced and inexperienced personnel. 
Technicians who had just completed maintenance training but who had no on-the-job 
experience solved the troubleshooting problems in about the same time as the more 
experienced subjects when they both used the trouble locators. 

Two additional Air Force studies provide further information on the extent to 
which improved job performance aids can compensate for training and C 3 q)erience. 
Folley and Hunger (19, p. 28-29) describe an unpublished study by Hoehn and 
Aukes which tested the effectiveness of a troubleshooting guide consisting of pro- 
cedural instructions for locating malfunctions in a simple electrical system. The 
major findings indicate that troubleshooting ability of untrained and partly trained 
men can be increased by procedural instructions. The gain appears to outweigh 
most of the three months of training in electrical principles and general trouble- 
shooting practices. Folley and Hunger (19, p. 7) also describe a study of Hoehn 
and Warden which indicated that Air Force technicians with a median of seven 
months experience did as well as technicians with a median of 21 months experience 
when both used an experimental troubleshooting guide. The less experienced tech- 
nicians using the guide employed more efficient procedures than did the C3q)erienced 
technicians who were using conventional manufacturer’s manuals. 

In the PIHO study cited in the previous section, definitive results have not yet 
been reported. Nevertheless, preliminary findings based upon the employment of 
a prototype audiovisual presentation aid suggest that such a device, along with re- 
formatted data, can compensate for training deficiencies. It is expected that a 
maintenance information presentation system will provide increased aircraft avail- 
ability and significant long term cost savings (37). 
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The ejqploratory research into basic problems of automating training for future 
information systems was aimed at the eventual development of alternatives to the | 

high cost of training operators for computer-based systems. Among the techniques \ 

i 

studied was that of display technique, i. e. , taped lectures and slides versus instruc- I 

tional manuals. One experiment using an abstract hardware model of a representa- j 

tive Air Force information system task demonstrated that a taped lecture with logic 

I 

flow diagrams on slides projected directly onto the control panel of a console could | 

I 

serve as a performance aid or teaching program. When the same logic diagrams | 

were employed in an instructional manual to supplement the text, training time was j 

longer than with the tape/slide program (7). \ 



ARMY RESEARCH AND APPLICATION 

Like the Air Force, the Army has been concerned with improving the efficiency 
of maintenance personnel performance. Hence, much of its efforts related to job 
performance aids has been directed toward developing improved maintenance sup- 
port data. Twg, major programs in recent years were carried out by the Human 
Resources Research Office (HumRRO) operating under contract with the Department 
of the Army. Task MAINTRAIN included research on the development of a main- 
tenance manual designed to permit trained technicians to troubleshoot complex 
electronic equipment faster and more accurately (43). An ejqperimental manual 
covering troubleshooting the Nike Ajax missile system and its test equipment was 
evaluated by means of an objective test using two groups of inexperienced techni- 
cians to locate malfunctions in the system. The experimental group using the im- 
proved manual accomplished troubleshooting substantially faster and more effec- 
tively than did the control group which used standard schematic and functional 
diagrams. 

The HumRRO research under task FORECAST involved a variety of related 
efforts including the development of new training content and improved training 
methods coupled with improved job methods, aids, devices, and data formats. 
Conclusions were drawn concerning the total FORECAST approach rather than the 
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effectiveness of individual aids. Major products of the FORECAST approach are a 
series of troubleshooting guides used for isolating system malfunctions based upon 
signal flow information. The job aids enable the repairman to do his own thinking, 
while still benefiting from the deductions made by experienced systems analysts who 
have previously structured the system in the optimum manner for troubleshooting. 
Results of the studies suggest that the FORECAST approach can produce technicians 
in less training time than that required with conventional techniques (46). 

The Army Missile Test and Evaluation Directorate conducted a study of the 
relative effectiveness of an experimental Audio-Visual Information System (A -VIS) 
when compared to standard technical manuals as a maintenance job aid (52) . The 
A -VIS machine evaluated was an experimental device containing schematics, block 
diagrams and printed instructions on film chips . The experimental model contained 
10,240 film chips plus 80 hours of audio instruction stored on magnetic tape. Selec- 
tion of a film chip is initiated by punching out a code number on a four column push- 
button selector . Visual information is displayed on two screens which can be used 
concurrently or independently. Audio information is also accessed by use of the 
control console utilizing an eleven -bit code on the film chip. 

The study also evaluated four other modes of presenting maintenance informa- 
tion: audio information system (AIS) which involved the audio portion of A-VIS, 
while the video portion was presented in book form; visual information system 
(VIS), in which the video portion of A-VIS was used while the A-VIS aural instruc- 
tions were presented in book form; programmed technical manuals (PTM), in which 
all of the A-VIS information was presented in book form; and finally technical 
manuals which included advance copies of engineering drawings and manuals as sub- 
stitutions for standard technical manuals. 

Based on maintenance performance on a group of test problems, it was con- 
cluded that A-VIS was superior to technical manuals as a maintenance aid. The 
major contribution to audiovisual superiority was in information format. All data 
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I 

programmed modes (AVIS, VIS, AIS, and PTM) were superior to technical manuals. j 

While the contribution made by an automatic video retrieval system was significant I 

I 

and enhanced the value of the programs in no case did the use of an automated audio i 

^ • I 

retrieval system improve the capability of the program. In fact, there was evidence ] 

I 

that the audio actually degraded the A-VIS and the AIS modes. The most effective j 

mode tested was the visual information system, while the least effective was the | 

technical manual mode. I 



The Army, on contract to HumRRO, has also explored the needs for managerial 
personnel aids that would be useful for junior officer performance and learning (1). 
Based on a survey and discussion with Army officers, 13 job areas and 34 specific 
job functions were identified where performance aids would be useful for minimizing 
the amount and difficulty of the job content that must be learned by an inexperienced 
officer. A suggested format for a handbook was developed covering what the inex- 
perienced unit officer needs to know about operational and system checks of elec- 
tronic equipment. It was concluded that, while existing manuals and directives 
provide helpful guidance, job aids in easily readable form are needed by the inex- 
perienced officer who is expected to perform in a short time. These aids should 
enable quick and easy access at the job location and should emphasize essential 
knowledge and skills required early in the job. 

Another type of managerial performance aid, nomograms, have been developed 
by the Army Personnel Research Office for use in evaluating manpower policy alter- 
natives (35). The nomograms facilitate the evaluation of alternative rotation poli- 
cies by showing the steady-state relationship between personnel allocation percent- 
ages overseas and in the U. S. for various tour lengths and selected assignment 
policy alternatives. Using the charts, management personnel can develop estimates 
of the effects of a given policy alternative without using a computer and without thie 
need for complex mathematical formulations. 
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NAVY RESEARCH AND APPLICATION 

The Navy in recent years has sponsored several studies which bear directly 
upon the content of the present report. In 1964, a laboratory model of a head-worn 
film strip viewer together with audio tape was evaluated as a vehicle for presenting 
programmed guidance in maintenance task performance (9), The head-worn device 
is a light-weight unit holding a 35 mm film strip of up to 75 frames. In use, the 
wearer's environment is visible to both eyes, while a virtual image of the film is 
presented to one eye using a semi-transparent mirror. Attending to one image or 
the other is accomplished by focusing the eyes on either an environmental object or 
the plane of the virtual image. The device was used in the laboratory to appraise 
its capability in relation to certain display and environmental variables. In addition, 
it was compared with a larger audiovisual device the effectiveness of which had been 
previously established. 

It was concluded that the head-worn device was usable under a wide range of 
viewing conditions. When the two audiovisual devices were compared as aids to 
aligning radar display equipment, they were found not to differ significantly in terms 
of alignment times. The head-worn device was preferred by six of the twelve parti- 
cipants in the study, with four preferring the comparison device and two expressing 
a preference for maintenance manuals. The results appeared to support the feas- 
ibility of the head-worn device for guiding the performance of specific tasks where 
there is insufficient space for larger displays and where operator mobility is re- 
quired during the use of the job aid. However, a number of specific difficulties 
were identified in using the device which should be corrected in future models prior 
to further evaluation. 

An experimental locator for use in troubleshooting a radio transceiver was 
developed and evaluated by Rigney and associates (40, 41). The fault locator is a 
small, circular, plastic job aid incorporating circuit-front panel relationships 
necessary for fault localization. The aid was designed to be used in checking out the 
receive and transmit modes of the equipment along with a number of sub -functions. 
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The technician notes symptoms of malfunction during the checkout, enters these 
symptoms on the aid, and looks up the indicated malfunctions in an accompanying 
manual. 

In one study (40), 20 radiomen after a three hour training session were given 
six troubleshooting problems to solve. The results showed that personnel who are 
relatively untrained in electronics maintenance could learn to use the job aid in a 
very short period of time and perform specific maintenance tasks such as fault 
localization that have been recognized as giving highly experienced electronics tech- 
nicians considerable difficulty. The success of the trouble locator appears to be 
due primarily to the fact that it presents a we}|, structured and standardized approach 
to symptom recognition and fault localization. Tlie authors point out that the job aid 
opens the way for non-maintenance personnel to share a part of the maintenance 
work load and thereby relieve the over burdened electronics technician. 
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The fault locator was also evaluated on 30 electronics technicians divided into 
one experimental group and two control groups (41). One group was given three 
hours special instruction in the operation of the radio transceiver and the use of the 
fault locator. A second group was given three hours of instruction in the radio trans- 
ceiver, but was not given any instruction regarding the fault locator; while a third 
group was given no special instructions regarding either the radio transceiver or the 
fault locator. All of the elctronics technicians were given the task of isolating mal- 
functions to a faulty component. The first group, the experimental group, had to 
use the fault locator to localize the problem; while the two control groups did not 
have use of the job aid. The results of the study showed that electronics technicians 
could learn to use the job aid in the three hour training program and could benefit 
from its use. While part of the success of the experimental group can be attributed 
to the special training in the operation and recognition of symptoms of the radio 
transceiver; nevertheless, the results of both this study and the previous study 
demonstrate the effectiveness of the fault locator as a maintenance job aid. It ap- 
pears that both radiomen, who are relatively untrained in electronics maintenance, 
and electronics technicians can be taught to use the fault locator in a very short 
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period of time and can both benefit from its use. 

In a similar vein, Hooprich and Steinemann (25) conducted an experiment to 
determine if a wallet —sized conversion chart would be a practical and convenient job 
aid for electronics technicians in performing measurement conversions. Three 
groups of Navy personnel served as subjects: 42 experienced technicians, 30 train- 
ees from an experimental electronics technician school, and 30 personnel with little 
or no electronics training. The criterion used to evaluate the job aid was a meas- 
urement conversion test. One-half of each group was allowed to use the conversion 
chart while taking the test, and the other half was not permitted to use any aid. The 
results indicate that, within the non-technncal and trainee groups, those subjects 
using the conversion chart did significantly better than those not using any conver- 
sion aid. Of particular importance is the fact that both of the trainee groups using 
the conversion chart did as well on the criterion test as experienced technicians. 
Questionnaire responses indicate that the subjects considered the conversion chart 
to be of value in the job or training situation after only brief instruction and prac- 
tice. Requests by the experienced technicians to retain copies of the conversion 
chart suggest a positive attitude toward the job aid and justify the belief that it 
would be useful for certain kinds of technical work. 

The Army FORECAST approach described previously was also applied and tested 
in the Navy environment (47). Following a FORECAST analysis of the Long Range 
Navigation (LORAN) System, a technical manual and training program were develop- 
ed containing the major FORECAST products described previously. The graduates 
of the FORECAST training program were compared with graduates of conventional 
LORAN maintenance courses on their ability to find malfunctioning parts in the equip- 
ment. The graduates of the FORECAST program identified approximately three 
times as many malfunctioning parts as did the graduates of conventional training. 
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CIVILIAN RESEARCH AND APPLICATION 

As pointed out earlier, the articles in the technical journals and trade publica- 
tions generally speaking consist of individual reports of experiences in developing 
or using specific job aid devices. A number of reports were authored by represen- 
tatives of organizations associated with the development and sale of such a device. 
Reports of research findings of the type discussed in the previous section are almost 
totally absent from the generally available civilian literature. Nevertheless, in 
order to provide a complete picture it is worthwhile to identify and review the re- 
ported experiences of job performance aids in industry. 

The literature reflecting industrial experience has been concentrated in the 
areas of assembly and inspection. The remainder of the relevant reports identified 
have been grouped together and will be discussed in the final portion of this section. 

Job Performance Aid Research on Inspection Tasks 

The major industrial research on job performance aids identified during the 
course of the present study was conducted by Teel and his associates. Unfortunately, 
the research does not appear to be reported widely, but rather is contained primar- 
ily in company publications. Teel, however, did report to the 1966 meeting of the 
American Psychological Association on a study conducted with Chaney (48). A 
four -hour training program and a set of visual aids were desiged to overcome de- 
ficiencies in the performance of machined parts inspectors. Effectiveness of the 
two approaches was evaluated singly and in combination, using three experimental 
groups and a control group. The criterion used was the percentage of true defects 
detected in a selected sample of machined parts. It was found that use of either 
training or visual aids resulted in large gains in the detection of objective defects; 
but the combination of training and visual aids resulted in the gTeatest improvement. 
Specifically, the use of training alone resulted in a 32% increase in objective defects 
detected; use of visual aids alone resulted in a 42% increase; and the combination 
of training and visual aids resulted in a 72% increase; while the control group per- 
formance did not change. 
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The improvements resulting from separate application of the two approaches, 
training and visual job performance aids, were not significantly different in terms 
of overall objective detection performance. The visual aids, however, were twice 
as effective as training in improving the detection of mislocated holes. These find- 
ings suggest that the relative effectiveness of the two approaches is dependent on 
the specific types of defects being inspected. 

The visual job aids studied by Teel and Chaney consisted of a series of simpli- 
fied drawings. Dimensions, tolerances, and inspection criteria for the character- 
istics inspected were located on the drawings to reduce the need for calculations 
and reference to other materials. Similar items were grouped together on individual 
pages in order to focus the inspector's attention on one type of defect at a time. 
Characteristics that were not to be inspected were omitted from the page. 

It was concluded that for maximum inspection accuracy both training and visual 
aids should be developed. According to the authors, whether to use training and/or 
visual aids for any specific application should be based upon a determination of the 
cost-effectiveness of the two methods. 




Several unpublished studies by this group also merit discussion (26). Harris 
conducted two studies evaluating the effectiveness of photographs as job aids for in- 
specting solder joints. In the first study, photographic inspection aids for use in 
inspection of solder joints were developed and evaluated. A comparison was made 
of the performance of a group of 18 inspectors when inspecting solder joints with 
and without the aids. It was concluded that the use of the job aids significantly in- 
creased the agreement among the inspectors on the acceptability or unacceptability 
of the solder joints. Use of the photographic aids resulted in a significant decrease 
in the percentage of good solder joints rejected but no significant change in percent- 
age of bad joints accepted. Because of the success of the visual aids in improving 
one of the most difficult of inspection areas, Harris suggests that photographic aids 
would likely improve the performance in other kinds of inspection. 
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In a related effort, Harris studied various characteristics of photographs for 
their value in communicating inspection standards to inspectors. Eight sets of 
photographs were developed, one set for each combination of characteristics. A 
group of 24 inspectors inspected solder connections, first without using photographs 
and then using certain combinations of the photographs. The results indicated that 
use of the photographs significantly reduced decision and diagnostic errors and in- 
creased decision and diagnostic agreement. Black and white photographs were 
found to be as effective as color photographs in inspecting the solder connections. 
Also, 4x5 inch photographs were found to be as effective as those 8 x 10 inches in 
size. The results further indicated that photographs of examples near each side of 
the acceptable /unacceptable borderline were more effective than photographs of 
examples far removed from the borderline. 



In another study, Sadler evaluated an overlay inspection aid designed for use in 
visual inspection of the photographic masks used in the production of circuit boards. 
Thirty subjects inspected a sample of these photo masks containing a variety of 
defects. Half the subjects used the job aid, while the remainder inspected without 
it. Use of the overlay inspection aid resulted in significant improvements in the 
detection of two of the three types of photo mask defects studied. Sadler concluded 
that the overlay technique provided a significant improvement over current proce- 
dures and recommended that it be adopted for use in photo mask inspection. 

Job Performance Aid Usage in Assembly and Inspection 

A total of seven articles covering the merits of audiovisual presentations as 
electronics assembly job aids were identified during the review of trade publications 
and technical journals (2, 3, 11, 21, 23, 27, 42). The articles were all published 
during the period between 1960 and 1963. These reports described the experiences 
of companies in applying the tape/ slide type of performance aid. 

The basic components of this audiovisual display include a 35 mm slide projec- 
tor with self-contained screen and a two-channel magnetic tape recorder with ear- 
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phone attachment. The components are mounted in a display case which resembles 
a portable TV unit. The display is equipped with front panel control knobs, buttons, 
and loading; positioned for convenient use by the operator. The programmed script 
(on one channel of the tape) and color slides are synchronized through a step-by-step 
process to provide the worker with the specific instruction and slide at the moment 
required. The sequencing of slide and script is accomplished by inserting a pulse 
code on one channel of the tape which triggers the projector at the appropriate time. 
The A-V "package" can be operated either automatically or switched to manual 
operation at the discretion of the operator. 

Variations to the basic display provide added convenience or capability. For 
instance, a foot pedal or switch may be added to interrupt the automatic sequence or 
to trigger viewer-tape playback. A projector capable of projecting pictures on a 
larger external screen as well as the built-in screen may be used to more clearly 
illustrate complex wiring. Music may be inserted on the tape during the time be- 
tween completion of instruction and completion of the process. Simplification is also 
offered by the A-V system in that it is possible to use only one of the two basic com- 
ponents , the slides or the tape. 

The claims made in these seven articles do not appear to be based upon con- 
trolled research studies. In any event, the absence of supporting information ex- 
plaining how the data were derived precludes anything approaching an adequate 
evaluation of the information. Of the seven articles, four dealt primarily, if not 
completely, with the experiences of the company which developed the original tape/ 
slide device for its own use and later became the major producer of the AV unit for 
industrial use. The remaining three articles cited the use of the audiovisual units 
in six other companies and presented the results in more or less detail from four 
of the organizations. 

Despite the obvious deficiencies in the information covered in this section, it is 
felt that the report would not be complete without an attempt to summarize the re- 
ported information. Accordingly, Table 2 presents a summary of the specific 
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claims for the five organizations covered in depth in the seven articles. While it 
must be emphasized that the figures in the table should be interpreted with caution, 
the nature of the specific reported advantages merits discussion. As indicated in 
the table , one or more of the organizations claimed sizeable reductions in five 
specific areas; 

• Learning time 

• Number of defects 

• Supervisory time 

• Inspection time 

• Assembly time 



The specific savings claims were accompanied by a number of supporting 
statements of perceived gains. Among these supporting claims are: 

1. Standard procedures developed and corrected. 

2. Paced instructions give operator a specific target for performance. 

3. Allows operator to proceed at own pace. 

4. Increased capability to perform complex work. 

5. Eliminated need for close scrutiny, coaching, and guidance by . 
instructor for complicated tasks. 

6. Increased worker versatility. 

7. More uniform product quality. 

8. Improved product quality. 

9. Inspectors using AV program aid consistently inspected to the same 
criteria. 

10. Supervision freed of instruction duties, more time for managerial 
responsibilities. 

11. Employee acceptance due to reduction in decision making and in- 
creased assurance of satisfactory performance. 

12. Employee appreciation of solitude and elimination of distracting 
sounds when using earphones. 
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TABLE 2 

SUMMARY OF CLAIMS REPORTED FOR AUDIOVISUAL PERFORMANCE AIDS* 



Reported Advantages 


Company Designation 


A 


B 


c 


D 


E 


Reduction in learning time 


80% 


50% 


50% 




45% 


Reduction in defects 




60% 




85% 


49% 






90% 








Reduction in supervisory time 






80% 


70% 




Reduction in inspection time 








85% 


50% 


Reduction in assembly time 




64% 






50% 



* Based on review of trade journal articles (Bibliography 
items 2, 3, 11, 21, 23, 27, 42) 
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On the other hand, a few precautionary statements and possible disadvantages 
were contained in the seven articles. In this category were the following comments: 

1. Technique best applied when work was complex, time consuming and 
critical in regard to assembly steps and quality of workmanship. 

2. Resentment toward rote process from particularly skilled operator. 

3. Inexperience in using system can lead to trouble. 

4. High engineering change rates, small quantity production, and simple 
or repetitive assembly steps limit value of AV programming. 









Application of Job Performance Aids in the Scientific and Engineering Work Areas j 

Even a casual review of the technical publications today reveals the wide-scale I 

impact that the computer is exerting upon the nature and content of technical work ^ 

1 

areas. While in many cases still in the promissory stage, development of com- j 

'1 

puter-based job aid display systems will very likely play a major role in influencing ^ 

the informational requirements of many technical jobs. Although a thorough analy- j 

sis of the computer and related instrumentation goes beyond the scope of the present I 

study, a few of the highlights can be presented. For a discussion of the overall i 

R&D instrumentation area and its effects upon the management of professional j 

i 

and technical personnel, see Bronson's recent article (8). . 



Advances in the field of man-computer interaction are being developed so 
swiftly that very few of the accomplishments have been described in the general 
literature. For the most part they are being reported in the limited distribution 
reports of ^i^arious governmental, university, and industrial laboratories. One of 
the latest reports on the potential of computers for extending man's problem-solving 
capabilities was provided by Newman (34). In reviewing the implications of com- 
puter-based information processing and display systems, he predicts that on-line 
computer systems will play a major role in the upcoming emphasis upon improving 
high-level human performance in executive planning and decision making. He sug- 
gests that we are a long way from determining the upper limit of man's intellectual 
ability for problem solving. According to Newman, by concentrating on providing 
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computer and display aids and extending his ability to manipulate data, man’s prob- \ 

lem solving and decision making abilities can be increased tremendously. 



There is a second category of reports in the technical and trade publications 
that bears very directly upon the job performance aid field. These reports deal 
with the development and application of individual aids such as conversion charts, 
calculating tables, nomographs, special purpose slide rules, etc. The number of 
new aids being developed and reported on and/ or marketed is frankly amazing. 

These aids cover a myriad of applications, and are primarily designed to eliminate 
the need for performing mathematical calculations in order to arrive at solutions 
to such problems as unit conversion, material requirements estimation, machine 
speed and feed calculations, and the like. 

Reflecting the growing activity in the field is an article by Kravitz (29) which 
suggests that when faced with frequent use of a few selected formulas, it may pay 
to design and build ones own special slide rule. The author indicates that, with 
such a job aid, much of the repetitive calculations can be avoided and more rapid 
answers developed by personnel with lesser mathematical ability. 

With similar objectives, Baum reported on the uses of nomography in a re- 
search laboratory (4). Among the advantages claimed for nomographs are: 

• Expedites routine calculations. Saves time and tedious effort. 

• Permits technical assistants to carry out computations whose 
bases are beyond their mathematical skills. Eliminates formulas 
and the need to look up values in tables. 

In Baum’s opinion, only eighth grade arithmetic is needed for using nomographs, 
in contrast to college level mathematics otherwise required. Because of the many 
benefits of nomograph usage, Baum urges that consideration be given to including 
nomography in the engineering curriculum. Whitehead (54) agrees with Baum on 
the usefulness of nomographs for engineers. He describes a simple procedure for 
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constructing a nomograph. 






Finally, the development of an audio job aid for flight control system checkout 
and troubleshooting was reported in a trade journal in 1964 (14). The device con- 
sists of a specially designed magnetic tape player and handheld control. A prere- 
corded tape contains a series of detailed troubleshooting instructions on one track, 
while another track contains the troubleshooting logic sequence in digital form. De- 
pending upon the results of each test item, the tape automatically proceeds in logical 
sequence until, according to the article, the malfunction is detected. The tape also 
refers the technician to a specific page in an accompanying manual which shows 
interconnecting wires, replacement procedures, and other information required. 

It is claimed that through the use of this device less skilled personnel can trouble- 
shoot the system successfully. At the time of the article an evaluation of the aid 
was reportedly being conducted. Recent correspondence with the manufacturer 
indicates that the device was not successful in its original application due to the 
following reasons: a large number of modifications in the complex new flight con- 
trol system, resulting in great difficulty keeping the tapes up to date; problems 
with unreliability of the job aid devices initially supplied by a subcontractor; and 
labor jurisdictional problems as the job aid was intended to be used by less skilled 
personnel to perform maintenance tasks which the skilled tradesmen successfully 
argued they should perform. The device is still being evaluated on a very limited 
basis in a military environment. While the company that developed the system, has 
faith in its potential, it has decided against further exploitation of the equipment 
due to slow development of its potential applications and the pressures of more ur- 
gent and profitable work. 
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FIELD SURVEY OF PERFORMANCE AID UTILIZATION 
IN ELECTRONICS ORGANIZATIONS 

SURVEY METHODOLOGY 
Data Gathering Format 

Basic data covering the types of job performance aids being employed in the 
electronic assembly field were obtained through interview's with plant personnel 
and by direct observation of assembly/fabrication areas. 

Development and validation of the basic data gathering forms and general con- 
tent of questions to be asked during in-plant interviews were accomplished within 
the Philco-Ford Western Development Laboratories (WDL) facility prior to initiat 
ing the field survey. Several combinations of forms and methods of interviewing 
were tried out in a number of WDL fabrication departments with the following re- 
sults: 

a. Attempts to structure the investigation with gross task descriptions 
as core material proved to be too time consuming and restrictive for 
the level of information desired. The major deficiency in this approach 
was that many tasks associated with a given work area might not be 
covered during the interview if specific types of assembly operations 
had not been performed recently. 

b. Employment of an entirely open-ended, relatively unstructured inter- 
view also proved to be unsatisfactory in that significant blocks of in- 
formation were often omitted and had to be obtained wdth follow-up 
contacts. This approach also presented problems in maintaining 

a central focus on job performance aids rather than on production 
methods, materials, etc. 
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c. 



A general outline of interview content that did not require a rigid 
sequence o ’ questions and answers proved to be the most efficient 
and comfortable technique to follow. The data gathering format de- 
cided upon served as a frame of reference for guiding the interview 
and was not intended to cover all the details of the entire interview. 
In keeping with the exploratory nature of the survey, the items on 
the form served as the stimulus for a variety of additional questions 
during the lengthy interview and observation. A copy of the inter- 
view guide employed is included as Appendix D. 



Selection of Companies 

A total of twelve industrial facilities were included in the field survey. Because 
of the exploratory nature of the study and the limited scope of the field survey, no 
attempt was made to secure a representative sampling of a specified population. 
Rather , the selection of firms was made on the basis of expected yield of practical 
information. Specific considerations in the selection of plants to be surveyed were 
as follows: 



Size of facility. An attempt was made to obtain as wide a range as possible 
within the primary geographical area of the survey. The number of employees 
directly involved with assembly tasks ranged from 50 to 1500. Estimated overall 
organization size ranged between 400 and 15,000 employees. 



Geographical location. The majority of facilities contacted are located in the 
San Francisco Bay Area bordered by San Francisco on the Northwest, Oakland on 
the Northeast, Santa Cruz on the Southwest and San Jose on the Southeast. This 
area covers approximately 1200 square miles and represents a population of over 
two million. Of the twelve facilities included in the study, eleven are located in the 
Bay Area and one in Los Angeles. 
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Type of manufacturer . Manufacturing processes include assembly of com- 
ponents on printed circuit (PC) boards, assembly of completed PC boards into 
sub-assemblies and, in turn, into final assemblies. These final assemblies varied 
from electronic desk calculators to complete missile and space vehicle communica- 
tion and control systems. Production processes covered by the survey included: 

r 

soldering solid state components on PC boards; construction of wiring harnesses 
for boards that contained as many as 8, 000 separate wires; and manufacture of 
integrated circuits (IC), also referred to as chips or dies. 

Experience with audiovisual aids . A number of firms were included in the 
survey because they had, at one time or another, employed various types of audio- 
visual devices for direct application to the assembly process. These applications 
involved the development of programmed procedures covering assembly tasks for 
a given work station. 

Job skills. The coverage of job skills paralleled that of manufacturing pro- 
cesses in that the number of basic skills increased with the number of, and diver- 
sity of, manufacturing processes covered in the sample. Basic skills and knowl- 
edges found during the survey were: soldering, wiring, manipulation of various 
microdevices employed in the manufacture and assembly of IC chips, inspection, 
and attachment of IC terminals to external electronic leads. 

Data Collection Procedures 

Personnel or industrial relations departments of firms selected for the survey 
were contacted initially by phone. The nature and purpose of the study was ex- 
plained and a referral was requested to the department or organization most closely 
associated with production or in-plant training areas. 

The next contact was with production or training personnel. The nature of the 
study was explained in detail and an appointment was requested for direct interview 
and tour of electronic production areas. All requests for referrals were granted. 
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Access was denied to the production areas of only two micrologic firms because of 
plant security policies invoked jto preserve proprietary manufacturing processes. 
Personnel contacted were highly cooperative in all phases of the investigation and 
provided more information and assistance than originally requested. 

The data collection method usually employed was to conduct the interview with 
two investigators; the first asking questions and engaging in the majority of con- 
versation, while the second recorded significant details. Interviews averaged two 
hours in length with an additional hour being devoted to a tour of production areas. 
Recorded notes were later transcribed, following a standard report format. Follow- 
up contacts by phone to verify information previously obtained, or to secure addi- 
tional information, were made in a number of instances. 



SUMMARY OF SELECTION AND TRAINING PRACTICES 
Selection 

The plants covered by the survey, with only two exceptions, have selected and 
trained inexperienced personnel for electronic assembly positions. Common selec- 
tion criteria are 10th grade education, normal color perception and average scores 
on some form of motor coordination, clerical perception, and form perception tests. 

I Training 

Pr e -e mpl oy ment training. Applicants meeting the general requirements must 
in most cases successfully complete a company sponsored 40 hour classroom train- 
ing course as a pre-condition of employment. These courses cover soldering tech- 
niques (or micro production techniques) , electronic component nomenclature, under- 
standing engineering drawings, and general familiarization with assembly techniques. 
At least 50% of the standard one week training course for entry level assemblers is 
devoted to acquisition of basic manipulative skills required for soldering or harness 
construction and parts handling techniques. The remainder of classroom time is 
devoted to familiarization of the trainee with parts nomenclature, production draw- 
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ings, and basic job performance aids. Applicants are typically not paid while in 
training status and are not guaranteed employment upon successful completion of 
the company sponsored course. The dropout/failure rate for these courses has 
been reported as being relatively small by firms covered in this survey. 



Post employment on-the-lob training. Personnel successfuly completing the 
pre-employment training courses are hired by the sponsoring firms with few excep- 
tioxis. Newly hired personnel are placed at an assembly station on the first day of 
employment and are expected to perform standard assembly tasks. A foreman or 
experienced worker usually conducts informal on-the-job training (OJT) on an as - 
required basis until the new worker is considered to be proficient. 

None of the firms contacted had maintained historical data that would indicate 
statistically derived learning curves for new employees. According to supervisory 
and staff personnel, from two to five days are required to fully meet production 
standards for the simplest assembly tasks. 



JOB PERFORMANCE AIDS USED IN ELECTRONICS ASSEMBLY 
Overview 

Several types of standard job performance aids were found to be in use in each 
facility contacted. These aids included engineering and special drawings, production 
prototype models, and f)hotographs of subassemblies and final assemblies in various 
stages of completion. Such aids are fairly common in industry and have been exten- 
sively employed throughout the development of the electronics field. 
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other types of aids such as audiovisual devices, audio devices and specially 
designed templates or overlays have also been applied to the manufacturing or as- 
sembly process. Results of the field survey reflecting the use of these various types 
of aids are given in Table 3. Unfortunately, objective data documenting the specific 
effects of job aids were generally not developed or were otherwise unobtainable 
during the survey. This was particularly the case with the more standard job aids 
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that have been in existence so long they are considered to be an intrinsic part of the 
job and their effectiveness is seldom, if ever, questioned. 

Visual Job Performance Aids 

It can be seen from Table 3 that the use of engineering drawings is universal 
within the sample studied and that employment of three dimensional models is fairly 
extensive. The engineering drawing must of necessity be employed during some 
phases of the manufacturing process since it is the official document or standard 
covering the component or assembly being produced. This type of job aid includes 
drawings of one or more views of the items being produced, accompanied by assem- 
bly instructions and lists of materials. 

The use of photographs, models, or specially prepared drawings normally 
served as augmentation material to the engineering drawings. These additional aids 
were usually developed when basic engineering drawings were found to be inadequate 
with respect to simplicity or degree of detail required for production purposes. The 
use of photos, models, or special drawings varied within a plant from production 
run to production run; their use being dependent upon such factors as complexity 
of assembly, type of assembly technique, required number of items being produced, 
and number of components or electrical connections required to be installed at an 
assembly station. 

The determination to use models or photos was usually made by industrial engi- 
neering or training personnel during the development process for a given production 
run. In several cases, models or photos were developed to alleviate excessive 
human error or reduce production time at given work stations. Several firms 
followed a standard practice of producing special drawings and assembly instructions 
for all assembly operations. They employed the general format of simplified, step- 
by-step instructions used by several manufacturers of "do-it-yourself" electronic 
component assembly kits. 
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Overlays developed by two of the companies have proved to be quite successful 
in the assembly of printed circuit cards containing a large number of solid state 
components. The arrangement of component bins has been integrated with the use of 
overlays to virtually eliminate operator decision in selecting or locating components 
to be attached to the printed card. One firm indicated that production efficiency has 
been "significantly increased" and that assembly error has been reduced to near zero. 
Use of the overlay devices also materially reduced familiarization time for workers 
newly assigned to printed circuit assembly stations. 



Au diovisual Job Performance Aids 

These types of aids have been used far less frequently than the types previously 
mentioned. Instructions for such devices are similar to that used in the special 
drawings and simplified instructions of the "do-it-yourself" electronics kits previous- 
ly discussed. Advantages of this system reported in the trade literature are that 
workers have continuous use of both hands through use of foot pedal controls and are 
provided with complete assembly instructions via the audio and video channels. A 
description of the audiovisual job aid was presented earlier in the report in the sec- 
tion discussing industrial job aid literature. 

Use of these devices as on-line performance aids accelerated rapidly during 
1960 and 1961. In seven of the twelve firms surveyed, approximatelj'^ 80 units were 
in use during the peak period of 1962. In general, they were withdrawn from the 
production area at nearly the same speed as their original installation. By 1963 very 
few remained in full time service as on-line aids. This extremely rapid acceptance 
of AV units reflected the pressure for quick answers regardless of the lack of objec- 
tive evidence. The need for careful planning and the importance of high quality audio- 
visual programs was either not recognized or was disregarded in the frantic rush. 
Moreover, the expectations of the amoimt of gains and the speed with which the gains 
would be realized may have been unrealistic. 



Out of a total of seven firms originally installing AV devices during the period 
from 1959 through 1962, four have continued their use in one form or another, 
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according to the field survey data. None of these devices was employed continuously 
for any extensive time period (a maximum of 6 months) as on-line procedural sup- 
port aids. The reason stated for this short term use by several firms v/as that em- 
ployees rapidly learned content and in most instances did not have to rely upon either 
the audio or visual portion after 5 or 6 trials. The period of time that a worker 
would have to rely on such information would naturally be dependent upon the length 
and complexity of the assembly process and worker retention capability. 

One of the firms no longer using AV devices reported that audiovisual programs 
were initially developed during the early 1960's for 8 ^ombly of standard printed 
circuit cards that had been in production for some ti ’ The devices were discon- 
tinued when production records failed to indicate gains in output or product quality. 
Printed circuit cards assembled with the AV job instruction process required an 
average of 30 minutes to complete and contained approximately 20 to 30 components. 
Considering the short cycle time and the number of components placed on the PC 
boards, such tasks would be classified as simple assembly tasks by contemporary 
standards. 



Applications of AV devices to relatively simple, short cycle, assembly tasks 
with little or no gain in output per unit time was cited as one of the primary causes 
for discontinuing AV equipment as on-line job performance aids by other firms in 
the sample. Several firms also cited lengthy time lags and excessive costs required 
to re-program AV material to reflect changes in production methods and components. 



Several individuals who were directly involved with development of audio scripts 
and production of slides for the early applications indicated that their initial efforts 
were somewhat clumsy and that lack of "professional" programming also could have 
contributed to the disappointing showing of certain production line applications. 

Data that would have reflected the relative impact in terms of increased productivity 
or quality were not collected during this period, at least in any consistent fashion. 

It might have been impossible to determine the effect of AV or similar devices at the 
time even if data were collected because of the many variables operating in the pro- 
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duction environment. 

The four firms continuing to use AV systems report that the devices are pri- 
marily used as storage libraries of assembly instructions for infrequently produced 
equipment that requires unique assembly practices and techniques. Employment of 
the devices in this manner eliminates the requirement for costly retraining of per- 
sonnel in preparation for production. Also reduced are rejections and re-work 
rates that are typically highest during the early phase of a production run. 

One firm also uses AV systems to maintain desired levels of skills and pro- 
duction practices for areas requiring high quality control standards. Programs 
covering basic techniques of potting or application of gold foil, for example, are 
periodically presented to workers in the production area. Direct relationships be- 
tween presentation of material and subsequent improvement of production quality 
is claimed for this application. 

Audio Job Performance Aid Devices 

Employment of audio job performance aids in the electronics industry has been 
relatively recent. The three firms indicated in Table 3 as using these devices have 
installed them within the last two years and are still in the process of experiment- 
ing with the technique. In all cases, audio job aids have been programmed for 
cable harness or cable board wiring that by its nature cannot be simplified or re- 
duced in complexity. Use of audio has proved to be highly successful in this appli- 
cation. Programmed audio instructions on tape at one plant reduced assembly time 
from an average of 17 hours to approximately 2-1/2 hours. Reduction in wiring 
error was also noticed, although specific data were not available. 

Taped instructions at a second plant covering a very lengthy and complex re- 
wiring of switchboard units have resulted in a 50% reduction in labor hours for wiring 
tasks. Along with labor hour reduction, errors averaging six per job have been re- 
duced to two. Additional manhour and cost savings were effected by the elimination 
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of troubleshooting time induced by wiring errors. A further advantage attributed 
to the audio system was that an inexperienced worker with only basic skills could 
be placed at the station and immediately begin wiring tasks without special familiar- 
ization training. 

Audio tape instructions have also been developed at one facility to cover instruc- 
tions for repair of equipment that is infrequently processed. Use of such instructions 
has eliminated the requirement for employee familiarization prior to the undertaking 
of infrequently performed and rather intricate repair or troubleshooting tasks. 
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FIELD SURVEY OF PATIENT CARE AREA 

SURVEY METHODOLOGY 
Interviewing Procedures 

Data covering the types of job performance aids utilized in patient care were 
obtained through interviews with hospital personnel and by direct observation of 
patient care areas . The interview guide employed for the electronic assembly 
field survey was also used in the patient care area. Again, the basic form was 
supplemented with additional questions during the course of the interview. 

Interviews conducted with hospital personnel were less structured than those 
conducted for the electronics portion of the survey. However, the method of inter- 
viewing using two investigators was generally followed. Notes obtained during the 
interviews were later transcribed following a standard report format. 





Survey Sample 

A total of twelve hospitals were included in this portion of the field survey . A 
number of considerations were made in the selection of hospital contacts. Vari- 
ables considered were: 

Size of facility. The majority of hospitals contacted were relatively large with 
respect to bed capacity with the exception of two rural hospitals . The bed capacity 
ranged from a low of 50 to a high of 1200; with the majority of hospitals having 
from 400 to 600 beds. 

G eographical location. Of the twelve hospitals in the survey, nine were located 
in the San Francisco Bay Area bordered by San Francisco to the North and San Jose 
to the South. One hospital was located in the greater Los Angeles area; while the 
two remaining hospitals were situated in smaller communities located approximately 

30 to 50 miles to the south of San Jose . 
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Type of hospital . An attempt was made to select a cross section of hospital 
sponsorship. Those sampled included university, religious, military. Veterans 
Administration, Public Health Service, rural community, rural regional, and 
county. One convalescent home was included in addition to the above. 

Job skills. The coverage of specific job and knowledge areas was centered 
about those positions directly involved with patient care. Most often included were 
the positions of registered nurse (RN) , licensed vocational nurse (L VN) , and nursing 
aide. 

Source of information. Interviews were conducted primarily with directors of 
nursing or with RN in-service instructors. Several ccntacts were with training 
representatives of personnel departments and with hospital administrators . Direct 
observation of patient care areas was made in ten of the twelve facilities . 

SELECTION AND TRAINING PRACTICES 
Position Responsibilities 

The following descriptions of duties and responsibilities summarize the patient 
care positions included in the survey sample. 

Nursing Aide . Under direct supervision of nursing staff, assists in the care 
of hospital patients by such tasks as: making beds under open, anesthetic, and 
occupied conditions; answering call bells; serving meals; bathing patients; and 
taking and recording temperature, pulse, and respiration rates. May apply various 
hot and cold therapy devices (compresses, packs, pads, etc.) and assist the RN in 
the routine portions of patient care . 

Licensed Vocational Nurse (LVN) . Assists in patient care under direction of 
physicians and professional nurses. Provides wide range of bedside care, as in- 
dicated under nursing aide, and assists RN with various therapy procedures. Charts 
patient’s records; applies dressings; operates inhalation therapy and other equip- 
ment; performs pre-operative and post-operative care; administers medications 
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with various restrictions. Most commonly, the LVN is permitted to administer 
previously prepared medications orally and by hypodermic syringe . 

The greatest variability in performance of duties and responsibilities was 
found in the LVN area. Responsibilities and duties were assigned on an individual 
basis in all hospitals employing LVN's. In-service and supervisory personnel 
reported wide variances in both training and abilities of LVN’s. Much of this was 
attributed to differences in licensing standards of various states in which licensing 
was originally granted. 

Registered Nurse (RN) . The general duty professional nurse provides the full 
range of nursing care, including administration of treatments and all forms of 
medication prescribed by physician. Prepares equipment and assists physician 
during examination and treatment. Observes, records, rnd reports patient's 
condition and reaction to drugs and treatment. May perform bedside care functions 
as described under LVN and nursing aide. May also serve as leader of nursing 
team composed of RN's, LVN's and nurses aides. 

Selection and Training Req uirements 

Nursing Aides . Selection requirements for entry level nursing aide personnel 
are typically: "some high school" and good mental and physical health. 

Training courses for entry level positions consist of from 50 to 100 hours of 
classroom training and from 150 to 230 hours of clinical experience (OJT) . Total 
training time ranges from four to eight weeks. Applicants with appropriate experi- 
ence are given standard orientation training which averaged 40 hours in length. 









.Licensed Vocational Nurse (LVN) . Vocational or practical nurses must be 
gradu ates of an instructional program in vocational nursing approved by the State 
and have passed the licensing examination. Training programs in the area surveyed 
are approximately one year in length and include both classroom study and clinical 
practice in basic patient care . 
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Registered Nurse. Must be registered as a professional nurse by the State 
of California. In order to be registered, an individual must be graduated from an 
accredited school of professional nursing approved by the State Board of Nursing 
Education and pass a State Board Examination; or, if registered in another state, 
submit evidence of having met equivalent requirements . Professional nursing 
education programs consist of (a) two year associate degree programs offered by 
junior colleges, (b) three year diploma programs conducted primarily by hospitals, 
and (c) four or five year baccalaureate degree programs offered by colleges and 
universities . 

All hospitals provide some form of orientation for newly hired LVN and RN 
personnel. The time periods devoted to this process range from one to two days 
for smaller hospitals to eight weeks for one of the government hospitals . The 
majority of courses are of one to two weeks duration. 

In-Service Training of Returning RN's 

Several hospitals provide formal retraining for RN's who have not practiced 
nursing within the past five years. Completion of a refresher course is an em- 
ployment requisite for five of the hospitals surveyed. These courses last from 9 
to 12 weeks and cover basic hospital practices and policies with particular emphasis 
being placed upon the newer procedures, equipment, and medications. In-service 
instructors provide the majority of training which normally consists of one-half 
classroom and one-half clinical practicum. Several instructors stated that this 
type of training must be handled on an individual basis because of wide variations 
in experience backgrounds of returnees. Individuals who have not practiced within 
the past 15 years must absorb a vast amount of information before they can resume 
basic floor nurse duties. 

Recruitment and Turnover 

None of the metropolitan hospitals covered in the survey reported any major 
difficulties with recruitment but indicated that turnover was a chronic problem . 
Annual turnover rate for combined patient care positions varied from 30 to 150 
percent. One hospital reported a yearly turnover of 100 percent for RN personnel. 

46 



o 

ERIC 



PHILCa 




WDL Division 



PHIUCO-FOHD CORPORATION 















WDL-TR3276 



No significant problems with availability of applicants have been encountered 
within the past six months . This current condition was attributed by several di- 
rectors of nursing to recent sizable salary increases granted to all levels of patient 
care personnel working in the San Francisco Bay Area. 

The reverse condition was experienced by the rural hospitals in the sample . 
Turnover was reported seldom to exceed 20 percent, but recruitment of a sufficient 
number of RN’s for full staffing was a continuing problem. 

JOB PERFORMANCE AIDS USED IN PATIENT CARE 



Overview 



The field survey of job performance aids for the patient care area covered 
those aids required to support RN's, LVN’s, and nurses aides. A number of 
procedural and reference types of job performance aids were provided to patient 
care personnel in all hospitals surveyed. These tj^pically included one or more 
procedures manuals, a standard drug reference, conversion scales, standing orders, 
and reference textbooks. Generally speaking, these job aids have been developed 
to support complex or critical tasks, such as preparing medications. Consequently, 
they are used primarily by professional nurses and, depending upon hospital policy, 
by LVN’s. The interviewing sessions and the limited observation of selected 
nursing stations were directed toward obtaining information on the types of job aids 
in existence. Objective records on their frequency of usage and their effect on job 
performance were not available in the sample of hospitals surveyed, and time did 
not allow obtaining such data in this exploratory study. However, some indications 
of the importance of job aids and the frequency of their usage are provided in the 
following paragraphs . 



Procedures Manuals 

Every hospital in the survey provided procedures manuals for each nursing 
station or patient care area. These manuals contain detailed procedures for all 
facets of patient care from admission through discharge. Organization and ar- 
rangement of the manual varied considerably within the sample. Some institutions 
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developed separate volumes for administrative, clerical, and technical procedures; 
while others arranged all procedures by functional area. Standard size, 3-ring 
binders and some form of tab indexing was used in nearly all of the manuals. 

These manuals are apparently used most heavily by newly assigned or inex- 
perienced RN and LVN personnel; the frequency being dependent upon the degree 
of supportive supervision given and the amount of time available for study. Several 
supervisory and in-service training personnel indicated that newly assigned RN's 
required from 2 to 4 weeks experience on a nursing station to become familiar with 
"hospital specific" procedures and policies. During this period the manuals were 
expected to be used frequently. No records were available as to how frequently 
procedures manuals were used or what effect they had on nursing performance. 
However, in-service training personnel often expressed concern that such aids 
were not used as often as they should be . 

Drug Reference 

One of more standard drug reference books were found at the nursing stations 
in every hospital surveyed . The most commonly used were a drug formulary and 
the Physicians Desk Reference (PDR). The drug formulary is a 6x9, tab indexed, 
loose leaf binder . Formularies varied in thickness from approximately 4 to 7 
inches. The PDR is a hard bound, thumb indexed, reference book that is approx- 
imately the size of a college standard dictionary. 

Both references contain basic information covering generic and trade names, 
normal dosages, contraindications and side effects for drugs in common usage. The 
formulary is more detailed than the PDR , containing additional information such as 
diagrams of molecular structures . 

Nursing personnel engaged in direct patient care stated that they used this 
type of reference from a minimum of once or twice a week to three or four times 
a shift. All RN personnel queried about use of this type of information stressed 
its importance and stated without exception that they should r efer to these references 

more often than they do. One of the problems is finding the time to locate the in- 
formation in the bulky reference volumes . The PDR was preferred to the drug 
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formulary in all instances where both were maintained. The reasons given for 
the preference being: (1) information access is easier and faster in the PDR; 
and (2) unnecessary technical information relating to detailed chemical and molec- 
ular structure is excluded from the PDR. 

Conversion Scales 

I . - — m • 

A wall chart containing conversion values from apothecary to metric systems 
was found on all nursing stations. This t 3 ^pe of chart was used frequently by all 
personnel who prepared medications . 

Other types of wall conversion charts were also found in niany of the medicine 
room or nursing station areas. One of the more common of these is a chart for 
converting drops per minute to cubic centimeters per hour, for use in connection 
with administering intravenous solutions . 

Many nursing personnel carried pocket conversion scales for personal use. 
In-service training staffs in several of the hospitals surveyed provided various 
types of these devices to new personnel. In all cases but one, the conversion 
scales were distributed by drug manufacturing concerns . The one exception was 
a government hospital that employed a conversion device of its own design. 

Several types of scales and conversions found to be commonly in use were: 

a) grain fraction to milligrams , 

b) grains to grams, 

c) apothecary to metric volume, 

d) centigrade to fahrenheit temperatures, and 

e) slide rules designed to provide special purpose calculations, such as de- 
termining desired rates of flow per unit time. 

A number of firms also produced cards containing basic reference information 
such as common hospital abbreviations, medical symbols, and normal values for 
blood, urine, and cerebrospinal fluid. 
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Standing Orders 

Standing orders; i.e. , procedures to be followed in preparing patients for 
various laboratory tests and x-ray examinations, were employed in all of the 
hospitals . These orders were usually printed on 3 x 5 cards and maintained in a 
central filing box at each nursing station. 



Cards covering specific tests scheduled for a patient are removed from the 
file and attached to the patient’s Rand (a universal term in local hospitals referring 
to a 5 X 8 visible card file system) at the time each test is scheduled. The card 
serves as an action ’’flag” and remains on the patient’s Rand until all required steps 
have been completed. A number of hospitals supplemented the standing order 
’’flags” with additional color coded indications that denoted ’’hold” or ’’add” actions 
for special dietary or medication requirements. 



Based upon our interview findings, this type of job aid is an integral part of 
the nursing routine and is universally used whenever patients are scheduled for 
tests requiring special patient preparation . 



Reference Textbooks 



Reference material consisting of medical and nursing textbooks and journals 
were also found at the majority of nursing stations. The availability of this type 
of material did not appear to be related to the type of patient care area involved, 
but in most instances reflected personal donations and availability of material from 
the hospital library. Several random spot checks of nursing information texts 
revealed publication dates ranging from 1954 to 1966. Frequency of use appeared 
to be highest in specialty stations such as intensive care and cardiac units. 
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DISCUSSION 

PRELIMmARY FRAMEWORK FOR THE FIELD OF JOB PERFORMANCE AIDS 

The results of the literature review and the field study, while limited in scope, 
nevertheless suggest that the area of job performance aids merits greater attention 
both in terms of conceptual development and in terms of application. 

Looking at the many different types and formats of performance aids and the 
variety of the uses to which they have been applied, it is interesting to speculate 
upon why greater progress has not been made in developing the field. Perhaps 
the very pervasiveness of job aids accounts for the lack of a unified conceptual 
approach. Being surrounded by informational aids of one sort or another, we have 
failed to recognize that, regardless of their dissimilarity, these aids share common 
objectives, common functions, and common problems . 

In contrast, the area of training has been recognized as a cohesive field and 
accordingly we see developing over many years a rather rich body of knowledge, 
basic concepts , and principles . Specialty areas continue to be identified where 
further knowledge must be obtained. This very structure in turn spawns additional 
activity which contributes further to the progress in exploiting training as an 
instrument of manpower utilization. 

Unfortunately, the same cannot be said for job performance aids. The major 
exception to this dearth of job aid research and application is in the area of 
maintaining complex equipment. Here, because of the critical problems associated 
with the operational utilization of electronic systems, and due primarily to the 
foresight of a group of military research and planning personnel, we can trace the 
progress being made in attempting to improve equipment maintenance through the 
design of improved informational aids . 

An excellent discussion of job performance aids and their role as stimulus con- 
trol instruments in man-machine systems has been provided by Wulff and Berry (55). 
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Nevertheless, much remains to be accomplished even in the area of equipment 
maintenance instructions. Cliapanis pointed out this problem recently (10) when 
he called attention to the language and words associated with tools, machines, 
systems, and operations as a large and important area almost entirely neglected. 

He suggests that changes in the words used in man-machine systems ”may produce 
greater improvements in performance than human engineering changes in the 
machine itself (10 p. 1). He calls attention to the need for a program of applied 
research; including such goals as the development of criteria for evaluating oper- 
ation and maintenance manuals, methods for testing them, and guides for developing 
them. Much of what Chapanis says about the language problems associated with 
man -machine systems applies directly to the overall problem of job performance 
aids discussed in the present report. 

This section will attempt to provide a basis for further study by (a) outlining 
a preliminary categorization of job performance aids and (b) discussing relationships 
between training and job performance aids . Obviously the present report can lay 
only a portion of the groundwork for such study . 

Bases for Categorizing Job Performance Aids 

A first step in bringing order to a potential field such as job aids is to suggest 
some organizational structure. Five possible bases for grouping and studying job 
performance aids can be identified as follows: 

(1) Impact on task content 

(2) Time of use 

(3) Information fxmction 

Information format 

(5) Sensory channel employed 

Impact on task content. One possible approach for categorizing job aids is in 
terms of how such aids alter the tasks which they are designed to aid. Two such 
classes are suggested: 

• Task supporting aids 

• Task modifying aids 
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Task supporting aids can be identified as those kinds of performance aids 
which guide the performance of a task but result in the same behaviors that would 
be performed without an aid. A procedural checklist is one example. In contrast, 
task modifying aids actually change task behaviors by allowing the job incumbent 
to achieve a desired result without the need for performing all of the steps that 
would be necessary without the aid. An example here is a nomograph or a table 
of square roots which provides the appropriate calculations without having to work 
through the mathematics. The distinction between these two classes of performance 
aids is discussed at some length by Wulff and Berry (55, p. 280). 

Time of use . A second possible classification of job performance aids could 

be based upon the time that such an aid is employed relative to task performance . 
Here we can see two possible distinctions in aids: 

• Off-line aids 

• On-line aids 

Off-line aids, or "reference aids" as Folley labels them (18,p.27), are aids 
which are used shortly before task performance. An example of this would be a set 
of instructions read by an employee before beginning a task and then remembered 
for a short period of time during task performance. On-line aids, on the other 
hand, are those which are referred to during the actual performance of the task. 
Folley refers to this class of aid as "cooperative" (18,p.27). 

Information function. Folley suggests that job aids can also be divided according 
to the kind of behavior processes aided (18, p. 27-28). Fo:* * example: 

• Information input 

• Information processing 

• Information output 

Job performance aids which support the information input process assist the 
individual in identifying and recognizing certain stimuli; e.g. , incoming signals 
or messages. Information processing aids are those used in making transformations 
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or calculations; such as conversion tables and special purpose slide rules. In- 
formation output aids can be visualized as those which support terminal behaviors, 
such as overlays which are used to indicate where certain electronic elements 
should be attached to circuit boards . 

Information format. The form in which information is presented is still a 
further basis for differentiating among job aids . While the variables of information 
format are many, the following appear particularly important to the design and 
utilization of job performance aids: 



• Verbal 

• Pictorial 

photo/drawing 

still/motion 

color /black and white 

• Numerical 

formulas 

• Model/mock-up 



Sensory channel employed. One of the most obvious distinctions among per- 
formance aids can be made in terms of sensory modality. While sensory channels 
such as tactile, and olfactory may someday be employed, the job performance aids 
in practical use today can be categorized in one of three groups. 



These three classes are obvious and need no explanation. The bases for de- 
ciding which channel to employ, however, are far from obvious. Information on 
which to base a decision is meager and at times conflicting. In one of the more 
recent studies of the area, Travers (51) raises serious question about the validity 
of many so-called principles of audiovisual presentation. He suggests that multiple 
sensory inputs may be of value only when the input rate is very slow. Similarly, 



• Visual 

• Auditory 

• Audiovisual 
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one of the research projects reviewed in the present study reported definite prob- 
lems when audio instructions were presented in conjunction with the visual presen- 
tation of an audiovisual job aid (52) . 

It is not intended, of course, that the tentative categories suggested here are 
in any way mutually exclusive. Rather, any single aid can be represented on all 
five classifications . Through such classification it is felt that progress can be 
made toward a better understanding of the functions of individual aids and their 
interrelationships in the overall field of job performance aids . 

RELATIONSHIP BETWEEN TRAINING AND JOB PERFORMANCE AIDS 

One of the major premises underlying the present study is that training and 
job performance aids are very closely related. Indeed, as illustrated previously 
in Figure 2, all three approaches - selection, training, and job aids - should be 
considered complementary tools in the performance development process. A 
prime objective in the design of certain job performance aids is to provide an alter- 
native to extensive training and thereby reduce costs and shorten the personnel 
development cycle. Furthermore, in the case of off-line aids discussed in the 
previous section, the objective can be viewed as one of providing systematic re- 
fresher or booster training on the job. That these efforts have met with some 
success is supported by the information obtained in the present study. 

A further indication of the intimate training-job aid relationship should be noted. 
No matter how simple to operate, the use of job performance aids must be taught to 
employees . Ideally such training should also foster employee acceptance and 
overcome aiiy resistance to change that may be exhibited, particularly by the older, 
more experienced workers . 

The development of an ideal personnel development system requires an iden- 
tification of behaviors that will be obtained primarily by selection, those that will 
be obtained by training, and those behaviors that will be obtained by job performance 
aids. This implies, of course, both the freedom for personnel planners to choose 
among various alternatives and the availability of criteria upon which to base such 
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decisions. In actual practice neither situation is common today. The study 
findings indicate, rather, that individual job aids were developed to solve some 
specific problem, such as excessive errors in job performance. Similarly, the 
criteria upon which to base such decisions are lacking. Hopefully, the decision 
to employ one or a combination of selection, training, or job aids will eventually 
be made on the basis of overall cost-effectiveness. 

While a great deal more information is required, the following considerations 
can be identified as important in determining the respective roles of training and 
job performance aids: 

Speed of performance. Tasks requiring particularly rapid performance; e.g. , 
emergency first aid, suggest training rather than procedural job aids. 

Task complexity. Tasks that are particularly complex and include many choice 
points favor job performance aids. Simple tasks, quickly learned, may not justify 
the costs of elaborate performance aids . 

F requency of performance. High frequency tasks suggest training emphasis. 
Infrequently occurring tasks imply job aids to offset the retention problem. 

Length of work cycle . Lengthy tasks lend themselves to performance aiding in 
preference to training. Short cycle tasks may not justify performance aid costs 
because of the speed with which individuals are able to perform from memory. 

Task stability. Tasks remaining stable over long periods of time suggest per- 
formance aids. Short lived tasks incur problems of performance aid modification. 

Task flexibility. Where established task sequences are critical, procedural 
job aids are necessary. Freedom to vary the style of task performance reduces 
the necessity of procedural aids . 
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Task precision. Tasks requiring extremely close tolerances imply consid- 
eration of job performance aids. Tasks in which extreme accuracy is not a re- 
quirement may not warrant application of aids . 





57 


taHILCO 

PHILCO-FOHD COnPOnATION 


WDL Division 






m 

i 



t 

j 







WDL-TR3276 







IMPACT OF JOB PERFORMANCE AIDS ON MANPOWER UTILIZATION ' 

The major objective of the present study was to examine the results that are 
available concerning job performance aids and, from this, assess their actual 
and potential impact upon manpower utilization. Of particular concern was the 
extent to which job performance aids can influence selection and training, quality 
of job performance, and productivity. Findings that bear upon these objectives 
have been derived both from a study of relevant reports and by field observation 
and interv5^';W. Because of the exploratory nature of the study and the embryonic 
stage of job performance aid development, the following comments must be in- 
terpreted more as implications rather than as firm conclusions . 

IMPACT ON TRAINING 

The military research on procedural maintenance aids definitely supports 
the contention that job performance aids can compensate for lack of training and 
experience. The single industrial research study bearing on this point also in- 
dicates that visual job aids can enhance inspector performance at least as much as 
training. Similarly in agreement are the claims made in the trade journals for 
tape/slide devices and their ability to reduce learning time. 

Generally speaking, the electronics assembly tasks surveyed required so 
little initial training that no deliberate attempt was made to reduce its length. 

Also significant is the fact that the organizations and individuals working in design 
and application of job performance aids represented methods engineering and 
quality control departments and, therefore, were not particularly interested in 
interfering in the basic entry level training which was the responsibility of the 
industrial relations department. 



Requirements for spec^^ic task training and refresher training of present 
employees, however, have been notably influenced by job aids. The experiences 
of a number of companies in the field study bear upon this issue, even though 
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their main goals were the improvement of performance quality and overall cost 
reduction. Results in the electronics assembly area indicate a nmnber of instances 
where reductions in training time were achieved with the introduction of various 
job aids . Impact of job aids upon training is particularly apparent in the case of 
infrequently produced equipment requiring unique practices . Preservation of 
complex assembly techniques for later usage can eliminate the requirement for 
costly retraining and further can effect reduction in errors and in rejections and 
rework costs. 

IMPACT ON SELECTION 

The findings just discussed suggest definite impact of job performance aids 
upon training. In contrast, the research literature reviewed does not provide 
a basis for drawing conclusions about the extent to which job performance aids 
can lower selection standards. This should not be interpreted necessarily as 
indicating that job aids cannot serve in this capacity; but rather that the motives 
of the investigators were not directed with this objective in mind. The research 
and application reports did not attempt to study this problem; consequently no 
specific results in this area could be expected. It is understandable, of course, 
that the military organizations sponsoring the research cited were much more 
interested in improving maintenance quality, and possibly shortening training 
time to enable more rapid availability of technicians , than they were in lowering 
overall selection standards . Nevertheless , one can infer from the results of two 
of the studies (25, 40) that lower -aptitude groups can improve their performance 
and, in certain tasks at least, equal the performance of higher level technicians. 

The results obtained from both the trade journal review and the field study also 
are lacking in specific information on the impact of performance aids on selection; 
again because there was no particular interest in exploring the problem. Clearly 
this is an area where additional information is required. 
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IMPACT ON JOB QUALITY 

Quality of job performance is expressed in a variety of ways; such as number 
and severity of defects, amount of inspection and rework time, level and amount 
of direct supervision and, finally, the amoimt of time that must be spent in re- 
medial on-the-job training. Military research studies explored the impact of 
maintenance aids on technician performance. The findings suggest that job per- 
formance aids can result in a significant reduction of errors and improvement in 
the number of malfunctions detected by experienced as well as inexperienced 
technicians . What little industrial research that has been done on job performance 
aids has been focused primarily upon the area of performance quality . The results 
clearly indicate that performance deficiencies of inspectors can be corrected by 
using specially designed visual aids during job performance. 




Similarly, the claims in the trade journals make a special point of citing I 

I 

company experiences that show ardiovisual aids have been a major influence in | 

reducing the number of defects and lowering the amount of time needed for close | 

supervision and inspection. The field study of electronics assembly also points | 

up the contribution that job performance aids can make to raising performance j 

I 

quality. 

I 

i 

IMPACT ON PRODUCTIVITY | 

i 

I 

While primary interest was directed toward reducing training time and im- j 

proving the quality of military maintenance, the impact of job aids on task per- i 

formance time was nevertheless studied in some of the military research on job ] 

aids. The findings indicate that use of job aids can save time, even on the part ^ 

of experienced technicians. Accordingly, it appears justified to infer that job . 

performance aids can affect productivity of maintenance personnel. 

! 

The industrial research study did hot touch upon this issue; however, the ] 

trade journal claims do make some reference to the reduction in assembly time. j 

Probably the reason that more publicity has not been given to the influence of per- 

i 

formance aids upon task performance time is the fear of criticism and emotional 
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reaction to the threat of task speed up . While some mention was made of an 
audiovisual device functioning as a pacer , this claim was overshadowed by the 
stress placed upon the provision for operator control over the pace at which job 
instructions are presented. The field survey likewise reflected a certain re- 
luctance on the part of job aid developers to label their products as speed-up 
devices per se. Nevertheless, in several cases the utilization of job aids resulted 
in a dramatic reduction in performance time of lengthy tasks . Savings have also 
been noted in the amount of troubleshooting and rework time as a consequence of 
fewer assembly errors when job aids are used. 

IMPLICATIONS FOR ROUTINE TASKS 

Despite the lack of an organized technology, the evidence accumulated in this 
study suggests a definite role for job performance aids in industry. Even where 
the tasks are routine, such as in assembly operations, experiences reflected in 
the trade literature as well as in the field study of electronics organizations sug- 
gest that job performance aids can have a significant effect upon job quality, 
productivity, and certain types of training. This effect, however, diminishes as 
jobs are subdivided into very short cycle, repetitive tasks. Under such conditions 
and with a very stable work force, the costs of job aid development may not be 
justified on the basis of gains achieved. When adequate job performance can be 
achieved in a matter of hours, for example, only a minimum of orientation training 
on the job may be all that is justified. Nevertheless, as jobs expand in scope and 
tasks increase in length and difficulty relative to the capability of the work force, 
the utilization of job performance aids bears careful examination. Job aids, also 
merit attention as skill upgrading becomes important. Depending upon how they 
are used, job performance aids can help to structure and formalize an otherwise 
ineffectual on-the-job training program. 

To what extent such aids can relieve the burden of training in overcoming 
basic deficiencies among the least talented of the manpower pool remains to be 
determined. Historically, the motivation for individual companies to explore the 
role of job performance aids in utilizing low talent manpower has been lacking. 

This is understandable so long as industry is able to recruit at the higher levels . 
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However, as the nation becomes more committed to utilizing the hard core un- 
employed, the extent to which performance aids can contribute should be deter- 
mined. 

IMPLICATIONS FOR TECHNICAL WORK AREAS 

While individual companies have not been particularly concerned with the 
problems of utilizing the lowest levels of the unskilled labor population, they 
have become painfully aware of problems in the skill shortage areas . How to 
satisfy the growing new requirements for skilled labor and how to forestall skill 
obsolescence of currently employed technicians and professional manpower has 
becom.e one of the industry's basic concerns. The extent to which the conventional 
solutions of raising selection standards and expanding continuing education can keep 
pace with new science and technology is open to serious question. 




In recognition of joint government-industry attention to this problem, repre- 
sentatives of the Department of Labor, the Office of Education, and all military 
services met with members of industry in June of 1966 at the first annual con- 
ference on Engineering Systems in Education and Training. In his ke 3 mote address, 
the Honorable Thomas Morris, Assistant Secretary of Defense (Manpower), out- 
lined five basic objectives. One objective is particularly germane to this study: 

"We need to expose more of our people to more information on an 'as needed' 
basis. With devices for information storage and retrieval, random access, 
and high speed communications - it seems to us that the time is fast approaching 
when knowledge can be more widely and immediately accessible - both for 
classroom training and for use on the job, thus minimizing the amount of 
knowledge which must be gained in the classroom itself." (37, p. 12-13) 

At this same meeting the Honorable Samuel Ganz, Deputy Manpower Ad- 
ministrator, Department of Labor, spoke of the need for accelerated training in 
the critical skill shortage areas (37, p. 160). This accelerated training approach 
would involve the use of new techniques in education technology coupled with the 
"type of work simplification methods used during World War II." Because of the 
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pressing need to utilize the seriously disadvantaged segment of the population, 

Mr. Ganz suggests consideration of redesigning positions in short supply and 
creating new, lower level positions . While not identified as such, a program of 
"accelerated training" could very well incorporate the benefits of applying in- 
formation technology in the job enviroment as suggested by Secretary Morris . 

As indicated throughout the present report, the military agencies have been 
particularly sensitive to the increasing gap between manpower resources and the 
requirements for technicians to operate and maintain increasingly complex 
man -machine systems. One potential solution that continues to be explored with 
growing emphasis, as Secretary Morris implied, is the technology for providing 
improved job -based communication at the point of use. 

The exploratory survey in the patient care area has provided one example of 
the limited usage of job performance aids in a critical skill area. While the 
relatively few information aids in use serve an important function, at this stage 
one can only speculate on the extent to which a comprehensive, formalized program 
of job performance aid development could demonstrate important gains in man- 
power utilization through improving human performance, reducing training prob- 
lems, and expanding the potential reservoir of applicants. Because of the range 
of skills represented in the health care field, a systematic program in this area 
could have significant impact not only upon this one critical area but upon many 
other work areas as well. 
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RECOMMENDED RESEARCH AND DEVELOPMENT 

The results of the present study demonstrate the potential of job performance 
aids. More information must be accumulated, of course, before the practical 
significance of job aids can be exploited fully. Two major areas of recommenda- 
tions are provided in the hope of contributing to the technical progress of the field. 
Specific recommendations for development activities are presented first, in 
keeping with the proper concern of the Labor Department for practical develop- 
ments in the field of manpower utilization. Following this is presented an outline 
of recommended research studies, the results of which would contribute to the 
long term growth and refinement of the job aid field. 



RECOMMENDED 



DEVELOPMENT 




The lack of a formalized job performance aid program in civilian work has 
been identified as a gap in the field of manpower utilization. Until such time as 
a systematic program is developed and demonstrated, the field will remain the 
hodgepodge that it is today; largely unproven in its potential claim as a tool for 
manpower utilization. Accordingly, it is recommended that such a program be 
undertaken within a skill shortage area involving a wide enough array of jobs 
and tasks such that the results of the program could be generalized to a broad range 

of task levels . 

Based upon the experience gained during this preliminary study effort, the 
patient care field could well serve as a model for such a developmental effort. 

The field is fast becoming a prime focus of concern as personnel shortages grow 
worse and the demands for quality health services increase. New patient care 
jobs are being explored to relieve the burden upon the overloaded professional 
and to provide the new skills required by the rapid increase in complex medical 
equipment and new techniques. Variations in education and training programs are 
being studied and expanded utilization of sub -professional personnel is gaining 
attention. The present usage of performance aids, limited as it is, suggests a 
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recognition of the role of external information assistance at various skill and 
professional levels. The critical importance of the patient care tasks and the 
need for extreme accuracy and attention to detail, coupled with problems of 
high personnel turnover and the need to improve the efficiency of personnel 
utilization, further argue for consideration of the patient care area as a ’’test bed” 
for job aid demonstration. 



While certain earlier applications of elaborate job aids met with only mixed 
success in the electronics assembly area, many applications have proved to be 
of value and new aids continue to be developed. It would appear that more in- 
tensive development activity is warranted as part of the nation’s overall program : 

to relieve skill shortages in the electronics area. The framework for the job I 

performance aid field suggested in the present report could serve as a basis for I 

developing and evaluating specific types of job aids for supporting electronics job 
behaviors. Findings in this study suggest that new applications could most profit- 
ably focus on complex, long cycle, precision tasks. Included in this program 
should be the development of functional characteristics and detailed design spec- 
ifications leading to the production and field evaluation of specific job performance 
aids . 

RECOMMENDED RESEARCH 

I 

The almost complete lack of job performance aid research in the civilian j 

sector indicates that a great many areas exist where new knowledge must be 
developed to support the long term growth of the job performance aid field. Four 
major questions are suggested as worthy of priority consideration: 






Besides the benefits that can be expected to accrue to the patient care field 
directly and to other work areas indirectly, the recommended program could 
develop and refine procedures and other guidance for use in later programs of 
job aid development. 
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1. What is the potential contribution of job performance aids for reducing 
basic aptitude requirements ? 



This question was suggested earlier as one deserving study as part of the 
nation's effort at improving the utilization of the "hard core" unemployed. Despite 
the pressure for immediate results in this area, it is believed that a research 
program into certain variables of job aid design should be conducted in order to 
provide a basis for future demonstration efforts. 



2. What is the ideal relationship between job performance aids and training 
aids? 



The apparent similarities and differences between training aids and job per- 
formance aids were identified in the present report. Much more remains to be 
done to explore the mutually supporting roles of these two techniques, the im- 
plications for functional and physical design characteristics, and the optimum 
integration of both techniques in an overall personnel development sequence . Such 
a study would be particularly helpful to the job performance aid field because of 
the knowledge that could be gained from the older, more developed training aid 
field. 



3. What is the optimum sensory channel for various types of job performance 
aids? 



The question of which sensory modality, or combination of senses, is best 
utilized in various performance aids is in need of solid experimental findings . 

The evidence as indicated in this report is far from conclusive and, in some cases, 
is contradictory . There is no question that information in this area will be of 
major, long term benefit in fostering an effective performance aid program. 



4. What are the implications of applying new information handling technology 
to the design of job performance aids ? 



A comprehensive study of new technical developments should form, a part of 
any long range program of job aid research. Specifically included in such a 
study should be an analysis of major advances in the techniques of information 
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organization, formatting, storage, retrieval, updating, and presentation. Fields 
of endeavor such as computer application, microfilming, closed circuit TV, video 
tape recording, and facsimile I'eproduction merit special attention as they may 
affect the future of job performance aid development and utilization. 
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SUGGESTED POLICY IMPLICATIONS 

It is premature to suggest firm policy on the basis of the exploratory research 
reported here. Certainly, the results of further study must be accumulated before 
any wide-scale policy changes could be implemented. At the same time, however, 
even the results of this one investigation can legitimately suggest directions for 
the anticipation of policy. 
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In attempting to develop a better imderstanding of the cyclical growth pattern | 

1 

of industrial AV performance aids and the general lack of progress in the area, | 

the investigators looked closely at the nature of the process by which relevsint j 

1 

job performance aid information is being disseminated. As a contrast to this | 

situation, the field of industrial training and its communication links were simi- 
larly analyzed. Underlying this activity was the hypothesis that the lack of gjrowth ] 

in the job aid field could be attributed, at least in part, to poor communication. i 

Stated more broadly, it was hypothesized that the growth of a technical field of i 

specialization is influenced by the number and quality of communication links j 

supporting the field. J 



o 
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The results of this preliminary analysis of communication processes in the 
job performance aid area as compared with the training field are shown in Figure 3. 
If we look at the upper portion of Figure 3, we can see that for the training field 
the nature and types of knowledge generators and communicators are indeed 
significant. Similar to many other professional areas, the training field is 
supported by several major independent sources of knowledge and transmitters 
of information; namely psychologists, educators, industrial training personnel, 
and manufacturers of training equipment and materials . Still another source of 
information is the professional training society; e.g. , American Society for 
Training and Development (ASTD) . While overlapping the other groups, these 
organizations serve as an entity to stimulate and communicate relevant information. 
Training societies, particularly at the local level, contribute significantly to knowl- 
edge dissemination by providing a vehicle for face-to-face communication. 
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Supporting the varied nature of the knowledge generators are a nmnber of 
different media available for communicating significant research and application 
results. It can be assumed that this rich communication network, with its oppor- 
tunity for claim and counterclaim, for theory and anti-theory, contributes not only 
to the transmission and application of new knowledge but also, through the feedback 
mechanism, directly fosters the generation of new knowledge. Particularly 
significant from the standpoint of the training man in industry, the lone practi- 
tioner can easily keep in the mainstream of new training developments . The dan- 
ger of repeatedly re-inventing the wheel is materially reduced in respect to the 
training field . One can hope with some degree of confidence that through this 
mechanism the progress of training technology moves ahead somewhat each year. 

Now in contrast, if we look at the lower portion of Figure 3, showing the 
job performance aid area and its communication network, we find an altogether 
different picture. Here, for all practical purposes, only two sources of knowledge 
exist — the individual job performance aid developers and the manufacturers of 
job performance aid equipment. Based upon our exploratory survey, it appears 
that individuals concerned with informational job aids are working in at least 
partial isolation. Their major source of information comes from very informal 
and infrequent contacts with their colleagues in other companies or from salesmen 
representing particular job aid devices. Our experience suggests that, because of 
the lack of information, it is not uncommon for industrial engineers in one company 
to be starting a job performance aid development effort without the benefit of knowl- 
edge from similar programs already completed by other engineers working in 
different companies . The need for more formal communication links was recog- 
nized by a number of the participants in our survey. 

Because of the lack of established communication channels, the salesman of 
job aid devices has become a rather key figure in the communication flow. With- 
out minimizing the services of the salesman, one can seriously question the 
situation where he becomes the major, if not the only, transmitter of technical 
information. Questions of objectivity and of communications effectiveness can 
certainly be raised. And yet, when no other channels exist, the investigators 
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have little choice but to rely upon the salesman to bring various company investi- 
gators into contact. In fact, during our field study, one of the respondents was 
surprised to receive separate requests from two other companies to share 
experiences in the use of job performance aids . Believing that the present report 
had been published and that this was the stimulus for such requests , he contacted 
the investigators for further information. After some discussion, it was agreed 
that very likely one of the salesmen had been instrumental in these requests for 
information sharing. 

The need for better and more extensive information was also reflected in the 
particularly enthusiastic cooperation the investigators received in nearly every 
company contacted. Requests to receive a copy of the survey report were almost 
universal. 

In the light of this dearth of professional communication in the job performance 
aid field and the probable negative impact upon technical progress , it is suggested 
that the Department of Labor give consideration to what positive role it might play 
in fostering improved communications and, in turn, greater progress in the field. 

It is hoped that reports of studies bearing upon job performance aids , such as this 
one, can be of some help in laying the groimdwork for this communication. Also, 
it is suggested that whenever a related study of manpower utilization, training, 
etc . , is undertaken under the Manpower Development & Training Act or related 
programs the investigators be encouraged to give some attention to the existence 
of job performance aids and the potential development of improved aids . The 
degree to which independent investigators recognize this area and include it as an 
item in their studies will contribute significantly to the objective evaluation of the 
strengths and deficiencies in industrial job aid development. 
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APPENDIX A 

ANNOTATED BIBLIOGRAPHY RELEVANT 
TO JOB PERFORMANCE AID FIELD 



1. Ammerman, H. L. Performance Aids for Junior Officers . i 

Alexandria, Virginia: The George Washington University, Human ? 

Resources Research Office, HumRRO, Technical Report 65-11, 1965. | 

% 

This study summarizes the comments and suggestions of air dei<mse battery \ 

officers concerning the types of managerial aids that would be useful for junior | 

officer performance and learning. A suggested format for a handbook was devel- S 

oped covering what the ine^erienced unit officer needs most to know about j 

operational and system checks of electronic equipment. A loose-leaf format i 

containing pocket size (5x8 inch) cards was recommended. The author concluded j 

that: (a) many small unit commanders in the air defense field desire additional 
job aids for newly assigned junior officers; (b) such aids should be designed for j 

quick and easy use at the job location and should emphasize essential skills and I 

knowledges required early in the job; and (c) existing guidance in manuals and j 

directives is helpful but easily readable summaries are needed for the very in- j 

experienced officer who is expected to perform as a manager of technicians soon j 

after assignment to a new job . ] 



2. "Audio-Visual Aids Join the Assembly Line." Supervisory Management, 

January 1963, 8, 51-2. 

This article relates how workers with meager skills and a minimum of 
training are doing complex assembly jobs with the aid of audio-visual machines. 
Basically , such systems employ the use of a slide projector with a screen and a 
tape recorder with earphone attachments; all of which are timed to the assembly 
job to be accomplished. This approached has been used primarily on production 
lines where complex electronic gear is assembled. The audiovisual system 
guides the worker through the process, with the slide projector visually depicting 
each step while the taped script describes the procedure . Keyed pulse codes on 
the tape s 5 mchronize the slides with the script and music is inserted in the script 
for time intervals between end of instruction and completion of process. According 
to the article, the average program of 15 minutes of tape and 30 slides will cover 
approximately one to one and a half hours of assembly work. It is claimed that 
assembly time can be cut by 50% with a 90% reduction in rejects. 



3. "Audio Visual Method Gives Faster, Better Production." Western Manu- 
facturing, January 1962, 23-25. 

The article describes the audiovisual (AV) instruction units for workers 
engaged in electronics manufacturing at a specific company. At the time of this 
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article, these units were used in 20 assembly programs and 18 inspection pro- 
grams varying in application from combined color slide/tape to slide or tape only. 
In actual electronic manufacturing at the plant, it was claimed that AV units have 
achieved saving in four major areas; learning time for assemblers and inspectors 
reduced by 45%, assembly time and inspection time by 50%, defects at inspection 
by 49%, and discrepancies in test by 44%. AV techniques as a communication tool 
were best applied when work was complex, time consuming, and critical in regard 
to assembly steps and quality of workmanship. Indications were that assembly 
output should exceed 35 units for optimum AV use and that high engineering change 
rates, small quantity production, and simple or repetitive assembly steps limited 
the value of AV programming. Results suggest that reduction in assembly time 
is a major source of cost savings . Improved operator performance stemmed from 
compulsory standardization in which the worker had no other information from 
which to work. The article pointed out that before an intelligent evaluation can 
be made by other manufacturers regarding the economics of using AV on the 
assembly line, they must consider complexity of assembly, quantities and durations 
of lot runs , potential quality and test problems , and labor turnover . 



4. Baum, S. Uses of Nomography in Modern Research. San Francisco: 

U. S. Naval Radiological Defense Laboratory, Reviews and Lectures No. 58, 
June 1958. 

The author discusses the use of nomographs (alignment charts) at the U. S. 
Naval Radiological Defense Laboratory. Specific examples are provided to il- 
lustrate the use of nomographs for expediting routine calculations and permitting 
technical assistants to carry out computations whose bases are beyond their 
mathematical skills. Among the advantages claimed are savings in time and pre- 
vention of errors . Only eighth grade arithmetic is said to be needed rather than 
college level mathematics otherwise required. Baum suggests considerations of 
including nomography in the engineering curiculum . 



5. Berkshire, J. R. Field Evaluation of a Troubleshooting Aid. Lackland 

Air Force Base, Texas: Air Force Personnel and Training Research Center, 
AFPTRC-TR-54-24, June 1954. (AD 537 25)* 



The development and preliminary evaluation of a set of troubleshooting 
aids are described by Berkshire . The aids consisted of color -coded schematic 
diagrams and written troubleshooting instructions called trouble locators . Results 
indicated that use of trouble locators reduced significantly the errors made by both 
experienced and inexperienced personnel and enabled inexperienced personnel to 
identify malfunctions in about the same time as the more experienced men. The 
procedures set forth in the trouble locators were apparently specific enough to make 
the color -coded schematics unnecessary for troubleshooting. 



Refers to the publication access number assigned by the Defense Documentation 
Center. References carrying such numbers can be obtained through the Clearing- 
house for Federal Scientific and Technical Information, Department of Commerce. 
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6. Bio-Dynamics Inc. Design and Use of Information Systems for Automated 
On-The-Job Training. Vol III; Experimental Use of Three Instructional , 

Concepts. L. G. Hanscom Field, Mass.: Electronic Systems Division, 

Air Force Systems Command, ESD-TDR -64-234, March 1965. (AD 616 544) 

This report is one of a series of five related reports dealing with the develop- 
ment of design principles for automated training subsystems which could be built 
into future information systems . Three experiments involving novel teaching 
concepts are described. The results of the experiments indicate that: (a) a 
teaching program ordered according to the discovery principle significantly reduced 
errors and performance time over that observed after training with a conventional 
training manual; (b) slides projected directly onto a control console, together with 
a taped lecture, were found to be an effective method of presenting an automated 
training program; and (c) graphical logical flow diagrams were found to be efficient 
instructions for teaching procedures for performing a querying -reasoning task. 



7. Bio-Dynamics Inc. Design and Use of Information Systems for Automated 

On-The-Job Training, Vol V - Final Report. L. G. Hanscom Field, Mass.: 
Decision Sciences Laboratory, Electronic fcsystems Divison, ESD-TDR -64-2 34^ 
April 1965. (AD 616 545) 

This is the final report of a series of five covering research on the develop- 
ment of design principles for automated on-the-job training for information system 
operators. The methods and results of the experiments detailed in the previous 
reports are summarized here. Conclusions and recommendations based on the 
overall research program are provided. One of the conclusions stated was that 
the slide/tape instruction programs can be used to provide on-the-job training or 
serve as performance aids with operational equipment. Since this technique can 
be evaluated in the field without requiring equipment modification, it was recom- 
mended that such a program be developed and used to meet an existing training 
need . 



8. Bronson, Z. "Instrumentation and the Management of R&D.” Research/ 

Development , August 1966, 23-36. 

In this article Bronson explores the subject of R&D instrumentation and its 
effect on scientific and technical manpower . The need for instrumentation is 
shown to generate forces which cut across the entire structure of R&D activity 
and influence both the rate and quality of progress . The study findings indicate 
"virtual unanimity of view that dependence on instrumentation is increasing and 
that it (1) can interfere with the freedom of research choices, (2) can involve 
changes in manpower requirements, and (3) can be the medium for promoting or 
inhibiting individual creativity and organizational productivity." 
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9. Brown, H. T. Evaluation of a Head -Worn Audio-Visual Aid. Fullerton, 

California: Ground Systems Group, Hughes Aircraft Company, FR 64-10-333, 
November 1964. 




A laboratory model of a head -worn film strip viewer together with audio tape \ 

was evaluated as a vehicle for presenting programmed audiovisual guidance in the \ 

performance of maintenance tasks . The head -worn device is a lightweight unit \ 

(approximately 30 oz) which holds a 35 mm film strip of up to 75 frames . The 1 

optical system includes a semitransparent mirror which allows the user to attend to I 

the reflected image or to look through the mirror to his environment. The film | 

viewer was used in the laboratory to assess its usability in relation to certain 
display and environmental variables, hi addition, the device was compared with \ 

a much larger (22 lb) audiovisual device whose effectiveness had been previously j 

established. WTiile some viewing conditions are more difficult than others, the I 

device was deemed useful under a wide range of conditions . Specific viewing | 

difficulties and means for offsetting them were identified and discussed. When the | 

two audiovisual devices were compared as aids to aligning radar display equipment, ] 

they were found not to differ significantly in terms of alignment times; only subjects | 

and trials produced statistically reliable performance differences. Of the 12 users, | 

6 preferred the head -worn device, 4 preferred the comparison device and 2 ] 

expressed a preference for maintenance manuals . Subject evaluations of the two j 

devices are presented along with a list of design improvements which the authors | 

recommend be incorporated in future versions of the head -worn device. 1 



j 

10. Chapanis, A. "Words, Words, Words." Human Factors, February 1965, 1 

7, 1-17. I 



The author calls attention to the area of language and the words attached to 
tools, machines, systems, and operations. He labels this area as almost entirely 
neglected and states that "changes in the words used in man -machine systems may 
produce greater improvements in performance than human engineering changes 
in the machine itself." Examples and illustrations of language problems are in- 
cluded to support his argument. A program of applied research on practical 
language problems is outlined . Among his research suggestions are a critical 
incident study of language problems encountered in man -machine systems; the devel- 
opment and experimental validation of principles for structuring short sentences ; 
and the development of methods and criteria for designing and evaluating instruc- 
tion manuals. 



11. "Col or -Sound Slides Speed Assembly." American Machinist/Metalworking 
Manufacturing, April 30, 1962, p 108-110. 

This article describes the Hughes Aircraft Videosonic system, how it is 
implemented and what the results are. The videosonic concept is described as 
going one step beyond the audiovisual approach in that it takes into account the 
assembler’s time and physical motion required to get parts and tools; thus pro- 
viding a "packaged" environment for each worker. In addition to the basic audio- 
visual equipment, each work station is equipped with adjustable parts bins, vise-type 
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work holders, and receptacles for all necessary tools and soldering irons. It 
was indicated that when assemblers were transferred from one job to another, the 
Videosonic method reduced retraining time from a month to a week to reach 80% 
or better efficiency. In addition, assemblers experienced about a 33% increase 
in individual productivity and a 60% or more reduction in rejects. Other x’esults 
included more uniform productivity, freeing of supervisory personnel from instruc- 
tion duties, and a 50% cut in learning time for assemblers. Further indications 
were that with its use, an unskilled assembler could achieve at least 80% efficiency 
on a new job in about a weeks training time and three days experience , as contrasted 
to more than 30 days for the same level of efficiency without Videosonic equipment. 
According to the article, a typical complex assembly of 3210 parts, previously 
requiring 160 hours to assemble, can be reduced to 65 hours of assembly time 
with Videosonic equipment. 



12 . 



Davis, R. M. Effective Technical Communications; Mecl^ical Descrip- 
tion - Experiment II. Wright -Patterson Air Force Base, Ohio; Air Force 
Institute of Technology, Technical Report 65-1, 1965. 



Results of a study to determine the relative effectiveness of different forms 
of written technical material are presented in the report. The effects of four 
variables upon written communication process were tested by varying the order and 
content of a technical description presented to four groups of young men classified 
as: bright with technical interest, bright without technical interest, average with 
technical interest, below average with technical interest. Subjects read material 
with varying content and completed a comprehensive test. Results indicated that: 
size and shape could best be presented by drawings, or drawings plus verbal 
descriptions for bright and average readers; while use of drawings to denote size 
and shape significantly reduced reading time for all groups . 



13. 



Day, W. F. and Beach, Barbara R. A Survey of the Research Literature 
Comparing the Visual and Auditory Pr esentation of Information. Wright- 
Patterson Air Force Base, Ohio: U.S. Air Force, Air Materiel Command, 
AF Tech Report 5921, November 1950. 



A literature survey, conducted under a USAF Air Materiel Command contract, 
was made of existing articles and studies relating to transmission of information 
via visual and auditory channels. Results of the survey, covering 34 articles 
dating from 1894 through 1946, indicated that the literature did not support the 
thesis that one system was superior to the other as a communication channel; with 
one -half the studies supporting visual presentation, and the other half supporting 
auditory. Evidence indicates the combined visual/auditory presentation to be 
generally most efficient. Factors influencing the use of one channel over the other 
are identified and areas for further investigation are suggested. 
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14. "Design Evolution Gains Maximum Utility." Aviation Week & Space Tech- 
nology, September 7, 1964, p 106-112. ' 



Included in this article is a description of the Smiths Aural Diagnostic In- j 

spection Equipment, known by the acronym of SADIE . The device was reported j 

being used by British European Airways for maintenance of the Smiths flight ] 

control system of the Trident jet transport and also being evaluated by the Royal I 

Air Force. SADIE consists of a small (10 lb), specially designed magnetic tape i 

player and hand-held controller. A prerecorded tape, housed in a cartridge which 1 

slips into the tape player, contains a series of detailed instructions for system i 

troubleshooting on one track, while another track on the same tape contains the | 

troubleshooting sequence logic, recorded in digital form. If the desired result I 

is observed, the chief maintenance man pushes the " 5 res" button on his small j 

hand-held controller and the tape automatically advances to the next test instruc- j 

tion. If the prescribed result is not obtained, the "no" button is pushed and the I 

tape advances to a different test instruction. A third button on the hand controller 
makes it possible to repeat any instruction. Depending upon the results of each | 

test, the tape automatically proceeds in logical sequence until it finally identifies 
the malfunctioning element. The tape also refers the maintenance man to a speci- 
fic page in a small printed manual which shows interconnecting wires , replacement i 

procedures, and similar information needed. It is stated that the use of the SADIE 
not only permits personnel with lower skills to troubleshoot complex systems 
successfully, but also tends to equalize the time required to locate any malfunction ^ 

in the system. Company experience indicates that SADIE permits the isolation of | 

any malfimction in the flight control system in less than 30 minutes. 



15. Duggar, B. C., Rosenberg, R. C., Sheridan, T. B., Mayer, Sylvia. Design 
^d Use of Information Systems for Automated On-The-Job Training - Vol IV 
'Graphical Symbology and Logic Diagrams for Use as Training AidsT 
Li. G. hanscom Field, Mass; Electronic Systems Division, Air Force 
Systems Command, ESD-TDR -64-234, January 1965. (AD 616 551) 

This report, the fourth in a series of five, describes the development of a j 

graphical symbology and logic diagraming technique for use as a training aid. 

Symbols and the logic diagraming technique were refined by application to several | 

tasks and were judged to be useful: (a) for supplementing written instruction 1 

manuals, (b) as a teaching aid without text, and (c) as a performance aid when j 

displayed on an operational console . I 



16. Elliott, T. K. Effect of Format and Detail of Job Performance Aids in 

Performing Simulated Troubleshooting Tasks. Wright -Patterson Air Force 
Base, Ohio: Aerospace Medical Research Laboratories, AMR,L-65-154, 

1966. (AD 629 992) 

The effects of performance aid format, performance aid detail, and subject 
aptitude on the performance of paper and pencil data flow analysis tasks were 
measured. Sixteen subjects were used in a 2 x 2 x 2 design. Eight of these subjects 
scored between the 75th and 95th percentile on the electronic aptitude index of the 
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Airman Qualifying Exam; while the other eight scored between the 40th and 60th | 

percentiles. Each subject received approximately 30 hours of training and prac- j 

tice, following which he was tested at one of the two levels of detail. Each subject | 

was tested for 9 hours with aids in the block diagram format and for 9 hours with j 

aids in the list structure format. Order of testing was counterbalanced across j 

subjects. Criterion measures were (1) number of problems attempted; (2) per- | 

centage of errorless localizations; (3) number of localization errors per problem; | 

(4) number of localization errors of exclusion per problem; and (5) percentage of i 

errorless isolations . Subject aptitude was found to have the greatest effect on the | 

accuracy with which subjects performed both localization and isolation tasks. Level j 

of detail had the greatest effect on their speed (low level of detail subjects solved | 

nearly twice as many problems as high level of detail subjects) . Performance was j 

better with the diagrams than with list structures . | 



17. Folley, J. D., Jr. A Preliminary Procedure for Systematically Designing 
Performance Aids. Wright -Patter son Air Force Base, Ohio: Aeronautical 
Systems Division, Air Force Systems Command, Report ASD 61-550, 

October’ 1961. (AD 270 868) 

A procedure is presented for use of human factors specialists to design 
performance aids pertaining to operating and maintaining man-machine systems 
in the military. The procedure is based on analysis of the kind of behavior re- 
quired of the task performer and, presiunably, could be generalized beyond 
military application. Although fairly detailed guidelines are given for parts of the 
procedure, considerable judgment is still required. The author recognized the 
procedure as untried and admittedly as having weaknesses; offering stopgap 
solutions to problems requiring more research and development. Four steps in 
the design of performance aids are presented: (1) identification of task elements 
for which aids should be provided - this involves describing the job and its com- 
ponent tasks, and analyzing each task element; (2) determination of. appropriate 
functional characteristics of aids for task elements - what the aid must do, not how 
it is done; (3) specification of the physical characteristics of the aids to carry out 
the functions - this involves combining the functional characteristics into appro- 
priate groupings and determining the most suitable method of performing each 
combination of functions; and (4) evaluation, modification, and updating of the 
aids - this ranges from a review of the steps taken in designing the aids to an 
empirical tryout with a mockup or actual equipment . 



18. Folley, J. D., Jr. Research Problems in the Design of Performance Aids . 

Wright -Patterson AFB, Ohio: Aeronautical Systems Division, Air Force 

Systems Command, ASD-TR 61-548, October 1961. (AD 270 866) 

Analyzing the information needs for the process of incorporating performance 
aids into a man-machine system is used as the vehicle for identifying significant 
problems in. need of solutions. The process is structured as (1) determining the 
need for performance aids, (2) determining how the aids should fill the need, (3) 
selecting or creating devices to fill the need, (4) evaluating the effectiveness of the 
aids , and (5) coordinating changes in aids with changes in the needs . The argument 



I 

] 

I 

i 

I 



i 

i 

I 

I 



I 



I 

i 



•J 




J 

A-7 



PHILCO 




WDL Division 



PHILCO-FOHO CORPORATION 



o 

ERIC 















WDL-TR3276 



is made that aids should be considered early in systems development for greatest 
contributions , but that they have utility at whatever stage of development they are 
designed. The most fundamental research problem is the need for a sound scheme 
for describing tasks and behaviors , in conjunction with development of a model of 
transfer of performance capability from task to task. Another highly important 
area for methodological research is the capability for human performance, equip- 
ment performance, and system performance to be specified and measured in 
directly comparable terms. A third methodological problem to be solved is that 
of measuring system effectiveness per unit of cost, with emphasis on including 
human factors in more quantitative terms than possible to date. The major problem 
to be solved by empirical research is to obtain data on the characteristics of human 
performance in realistic situations. The emphasis here is placed on multivariate 
studies to determine the relationships among significant variables that affect per- 
formance. A longitudinal approach is suggested, in which a few subjects are 
studied for long term periods as they become highly skilled on some tasks . 



19. Folley, J. D., Jr. and Munger, Sara J. A Review of the Literature on De- 
sign of Informational Job Performance Aids . Wright -Patterson Air Force 
Base, Ohio; Aeronautical Systems Division, Air Force Systems Command 
ASD Tech Report 61-549, October 1961. (AD 270 867) 

Nine sources of military reports and psychological journals were surveyed 
for references pertaining to research on the design of performance aids. The in- 
formation from a review of 57 relevant references is presented as an annotated 
bibliography with a general discussion on the findings. The major portion of re- 
search reviewed was performed for the Air Force and was completed prior to 1958. 
None of the studies attempted to identify the significant variables to be considered 
in the design of aids . The preponderance of the work was concerned with developing 
and trying sample aids . The tryout of specific aids yielded limited information 
about the relationships among the conditions that were manipulated and measured. 

The general discussion of findings was organized under the headings; "Procedures 
for designing performance aids," "Determining the functional characteristics of 
aids," and "Determining physical characteristics of aids." The planned headings 
of "Determining the need or desirability of aids," "Evaluating the effectiveness of 
aids," and "Coordinating changes in aids with evolving changes in job structure" 
were omitted because not enough information was available on these topics . Major 
emphasis was given to studies using printed performance aid material, although 
some studies report on the use of auditory aids. Performance aids for simplification 
of troubleshooting have received the most attention, showing that procedural aids 
can facilitate such performance . The extent to which other kinds of aids can 
facilitate performance of other kinds of tasks is unkown. Useful guides for per- 
formance aid design were found, but explicit methods for designing aids were 
lacking. No adequate solutions to the problem of determining information require- 
ments of a task were available, and little guidance was found for making tradeoffs 
between information in the performance aid and information learned during training. 
It is suggested that more consideration be given to establishing information about 
human performance; such as definitive statements about the conditions under which 
errors are likely to occur . 
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20. Folley, J. D. , Jr. and Shettel, H. H. Tryout of a Preliminary Procedure 

f or Systematically Designing Performance Aids. Wright -Patterson Air 

Force jjase, umo: 6570th Aerospace Medical Research Laboratories, 

MRL-TDR -62-20, April 1962. (AD 283 605) 

The performance aids recommended by applying the design procedure to a 
sample of task data for an operational missile system were compared with those 
recommended on the basis of observation and analysis of task performance in the 
operational situation. Detailed application of the procedure was made for a small 
number of the 135 troubleshooting tasks that had been selected . It was concluded 
that, for the most part, the performance aids recommended by using the procedure 
would meet the needs of the operational situations quite well . One problem obscuring 
the evaluation was the fact that the level of performance in the field was not ade- 
quate to meet system requirements . It was also noted that the personnel for whom 
the performance aids were designed rejected the idea that their jobs could be 
proceduralized; yet they proceeded to develop procedures comparable to those 
produced by the task analysts . The major change made in the procedure as a 
result of the tryout was to require performance aids for every task in a system, 
except for tasks in which use of the aid would result in too slow performance . 

Several recommendations were made for performance aids to be used v/ith the 
missile system, including a cross -index file to help locate the appropriate proce- 
dural checklist. Three research problems were identified: (1) What is optimal 
format and level of detail for a procedural task checklist? (2) What combination of 
performance aids and training is most effective for developing troubleshooting 
ability? (3) By what means can behaviorally critical aspects of tasks be identified? 

21. Harker, W. A. "Audio -Visual Learning - Ifs More Than Hear-Sayl " Elec- 
tronic Industries. August 1961, p 103-105. 

This article describes the function, application, and results of the Videosonic 
system developed by Hughes Aircraft Co. Consisting of both information programing 
and display equipment (resembling a portable TV set) , the AV system was designed 
primarily for communicating complex information to electronic assembly line 
workers. Where used for on-the-job instructions, properly programmed material 
results in accurate and concise task definition which shows and tells exactly how 
each step is to be performed, in sequence, and at the precise moment it is to be 
done. Thus, the most complex of operations can be easily understood. For 
example, learning time for a new operator normally requires an average of 10 
times the allowed standard time to produce the first unit. It was stated that Video- 
sonic aids in reducing this to 4-1/2 times the standard for the first unit with third 
units produced at the standard rate. Immediate increases in productivity of 70% 
are not unusual. The most striking results, according to the author, are improved 
quality and reliability in workmanship and products. Where an average of five and 
one-half quality defects per unit had been experienced, a reduction to one-half 
defect per unit was achieved upon introduction of the system . 
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22. Henneman, R. H. & Long, E. R. A Comparison of the Visual and Auditor;|^ 
Senses as Channels for Data Presentation. Wright Air Development Center, 
WADC Tech Report 54-363, August 1954. (AD 61 558) 

This study was undertaken to provide communication engineers and psychol- 
ogists with information that might be used to support auditor y/visual trade-off 
decisions in equipment design. A taxonomy was developed to permit multiple 
classification of results and conclusions derived from a survey of literature rel- 
evant to the study. The authors state that the question of relative superiority of 
vision vs audition for communication channels cannot be answered without first 
determining the conditions and purposes. Variables to be considered are: types 
of responses; e.g. , comparison, quantitative, etc. ; operator variables; e.g. , 
previous habits; and environmental constraint; e.g. , ambient noise, excessive 
vibration, etc. Suggested applications for visual auditory either visual or auditory, 
and dual audiovisual presentation of information are presented in tabular form 
with accompanying estimates of confidencje based upon applicable research. 



23. Hill, D. A. & Tamsen, J. J. "Videosonic System Instructions Raise 

Quality Standards.” Industrial Quality Control, July 1961, 15-20 

In an effort to prevent or diminish human error during assembly, Hughes 
Aircraft Co. developed the Videosonic system (an audiovisual 35 mm color slide 
and tape display). In describing its development, application, and results, the 
article identifies this system as ”one of the most powerful tools every developed 
for communicating intelligence in such a manner as to assure accuracy , guarantee 
standardization, and facilitate the understanding of the most complex data to indi- 
viduals from all walks of life and of varying abilities.” It offers flexibility in that 
presentations may be used for long or short run projects, retained, and when 
needed again are implemented without the long indoctrination time required to 
retrain a worker . Noted improvements due to its use have been in the general 
areas of reliability, quality, production, and employee relations. At the time of 
the article, the Falcon Missile at Tucson Arizona was assembled, inspected, 
tested, and supported in the field through this system. A unique application of 
the system was its use in filling out employment questionnaires and security 
clearance applications; thus replacing a somewhat complicated instruction sheet. 

In regard to reliability, a 50 to 60% first-test acceptance ratio increased to 
between 75 and 85% on all electronic packages submitted. The most complex chassis 
in the Falcon Missile increased its first test average from 50% acceptance to 84% 
acceptance with Videosonic application. In regard to cost savings expressed in 
terms of costs expended and costs saved, Hughes experienced a crossover point 
in the third year with savings exceeding costs during the following years. In total, 
the authors claim that the Videosonic system has improved communications , 
reduced costs , provided ability to adapt quickly to new market situations , and has 
greatly improved the reliability and quality of products . 
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Hoehn, A. J. & Lumsdaine, A. A. Design and Use of Job Aids for Communi- 
cating Technical Information. Lowry AFB, Colorado: AF Personnel and 
Training Research Center, AFPTRC-TR-58-7, January 1958. (AD 152 109) 



This report summarizes a number of research and development efforts on 
the design and utilization of job aids undertaken by the Maintenance Laboratory 
of the Air Force Personnel & Training Research Center. The primary objective 
of the research was to identify and test the effectiveness of improved techniques 
and procedures for the design of job instructions , or informational job aids , that 
could reduce the amount of training required for technicians and increase the 
efficiency of their job performance. Prototype materials and methods for the main- 
tenance of flight control systems and their components were formulated and 
tested; several experiments on factors affecting the comprehension of printed verbal 
instructions were conducted; exploratory research on audio presentation of instruc- 
tions was accomplished; and the design and fabrication of a prototype model of a 
special-purpose magazine loaded sound film projector was completed. Among 
the conclusions drawn were: (a) carefully designed job aids can reduce training 
requirements, improve the utilization of lower skilled airmen, and increase the 
reliability of their performance ; (b) for maximum efficiency in the development 
of maintenance capabilities , procedures should be developed to plan and implement 
job aid materials in a manner closely integrated with training materials; (c) proce- 
duralized, step-by-step instructions have been constructed so as to minimize 
requirements for advanced training even for such difficult tasks as troubleshooting 
of quite complex equipments; (d) better methods of job -task-analysis emphasizing 
the behavior processes which the job aid materials are to support and the conditions 
which foster these behavior processes are required for designing of job aids; 

(e) an automatic sound film viewer was demonstrated as technically feasible, and 
a preliminary study of a prototype of the projector indicated that carefully designed 
film material presented by such a device can enable airmen to perform lengthy 
and exacting bench check operations accurately with little or no previous instruction 
or practice . 



25. 



Hooprich, E. A. & Steinemann, J. H. An Investigation of the Utility of a 
Conversion Chart as a Job Aid for Electronics Technicians . San Diego: 



U. S. Naval Personnel Research Activity, Technical Bulletin STB 66-13, 
November 1965. (AD 624 608) 



As a part of continuing research to increase the effectiveness of Navy training, 
an experiment was conducted to determine if a wallet-sized conversion chart would 
be a practical and convenient job aid for electronics technicians in training and on 
the job. Three groups of Navy personnel were used as subjects: 42 experienced 
technicians, 30 experimental ET School trainees, and 30 personnel with little or 
no electronics training. The criterion used to evaluate the effectiveness of the job 
aid was a 10 item measurement conversion test. One-half of each experimental 
group was allowed to use the conversion chart while taking the test, while the other 
half was not permitted to use any aid. The major finding was that those trainees 
using the conversion chart did significantly better on the test than those trainees 
not using it, and they also did as well as experienced technicians using or not using 
the chart. Questionnaire responses indicated that the subjects considered the 
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conversion chart to be better than other similar aids and to be of value in the job | 

or training situation after only brief instruction and practice. Distribution of the | 

chart to Navy technical personnel was recommended, and proposed formats for the j 

cards were presented. j 



26. Industrial Applications Human Factors Research Abstracts. Autonetics j 

Division of North American Aviation, Inc., P5“1698/3111, 1966. | 

•i 

This is a company publication containing abstracts of 18 reports dealing with | 

human factors studies having industrial applications issued between September 1963 | 

and April 1966. The majority of the abstracts deal with studies in the quality | 

control/inspection area. I 



27. Jacobsen, T. L. & Williams, R. A. "Audio-Visual Concepts in Industrial 

Manufacturing." Mechanical Engineering, August 1963, 85, 46-49. 

This report discusses the benefits of using audiovisual aids in the manufac- 
turing process . Among the benefits cited were reduction in training time for new 
employees, removing the training burden from experienced assemblers, and 
broadening the scope of duties that assemblers are able to perform. Uniform 
procedures can be developed with the same quality control criteria displayed to both 
assemblers and inspectors. To allow for learning that takes place on the job, 
Jacobsen describes the use of three different audio instruction tapes (a) training 
tape with complete instructions; (b) standard tape with condensed instructions; 
and (c) "bikini" tape having only the critical precautions. Performance with this 
last tape is envisioned as a prestige symbol and a motivating factor. 



28. Kaplan, D. H. "Considerations in the Design and Installation of Cockpit 

Checklists , Placards and Manuals .*" Aero. Engineering Review , September 
1951, 10, 10-16. 

This article is concerned with the design, installation, and adequate utilization 
of cockpit checklists, placards, and manuals. A discussion based on relevant 
research is presented in terms of the following aspects of the problem; location, 
legibility (brightness, visual angle, observation interval), illumination under night 
visual conditions, the variations in checklists and manuals. A series of specific 
recommendations is offered concerning design, utilization, and experimentation 
with checklists , placards , and manuals . 



29. Kravitz, S. "For That Special Problem, Design Your Own Slide Rule." 

Product Engineering, February 28, 1966, 37, 74-77. 

The author suggests that where a few selected formulas are frequently used 
it may pay to design and even build a special slide rule. As a job aid, the use of a 
special slide rule can avoid much of the repetitive calculations, produce more 
rapid answers, and in some cases obtain more accurate values than with a standard 
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slide rule. Normally, a lesser degree of skill and mathematical capability are 
required to operate the special slide rule. The key question is whether the formula 
lends itself to a slide rule solution. The article deals at some length in formula 
conversion plus a new mathematical technique for positively determining whether 
or not the formula can be put into slide rule form. A description of how to build 
a slide rule and information on where to obtain special slide rules are also provided. 



30. 



Losee, J. E., Allen, R. H. , Stroud, J. W., Ver Huist, J. A Study of the 
Air Force Maintenance Technical Data System. Wright -Patters on Air Force 
Base, Ohio; 6570th Aerospace Medical Research Laboratories, Aerospace 
Medical Division, AMRL-TDR-62-85, August 1962. 



This research involved an extensive literature search and review of applicable 
regulations, specifications, and selected technical orders; visits to Air Force 
installations and agencies to interview management and operating personnel and to 
survey a large representative group of Air Force maintenance technicians; a 
survey of technical publication sections of contractors; and an experimental eval- 
uation of troubleshooting presentations . The findings highlight the impact of 
management on the procurement of accurate, timely, and economical data and 
identify areas in which management was found to be deficient. The report points 
out specific shortcomings in the data, their presentation, distribution, and use. It 
is observed that the eventual solution to the problem of maintaining the most complex 
system may well be automatic collection, storage, and retrieval of troubleshooting 
and repair mformation. Twenty recommendations are proposed to improve man- 
agement of the overall Air Force Technical Order system and to enhance the 
quality, usefulness, and timeliness of the data produced. 



31. 



Lumsdaine, A. A. & Hoehn, A. J. Behavioral Guides for Improving Utili- 
zation and Job Performance of Technical Personnel. Paper presented to the 
65th Annual Convention of the American Psychological Association, September 
1957. 



Improvements in communicating job information through specifically designed 
handbooks, audiovisual media, and other forms of "behavioral guides" are described 
as capable of improving job efficiency and conserving skilled manpower by expanding 
the range of technical activities requiring only minimum formal training. These 
performance aids provide simplified, standardized, step-by-step instructions for 
performing exacting troubleshooting, alignment and related operations. Illustrations 
of such guides are presented, along with preliminary data on resulting increases in 
job efficiency and suggestions for research to improve the design and use of be- 
havioral guides. 



32. 



Maloney, R. T. Portable Microvisual Systems. Berkeley, California: 
Lederer, Street and Zeus, 1966. 



This book deals with the retrieval of pictorial information using portable 
systems . Major emphasis is directed toward the description of portable microfilm 
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systems; however, a description of videotape and facsimile systems is also pro- 
vided. Applications of a number of portable microvisual systems are described. 



33. National Security Industrial Association. The Modern Technical Manual. 

Proceedings of January 1964 West Coast Meeting, Technical Publications 

Panel, NSIA Maintenance Advisory Committee. 

This is a report on a symposium held to direct attention to the imperative 
need to effect a major cost reduction in maintenance of military weapons through 
a revolutionary change in the concept, character, form, and function of technical 
manuals. A major contributor to the symposium, G. L. Belleville, reported on 
new methods and processes for storage, retrieval, and display of technical manual 
data at the point of use. The devices discussed included rear projection units, 
hard-copy printers, TV monitors, computer printout equipment, and portable 
readers . 



34. Newman, J. R. Extension of Human Capability Through Information Proc- 
essing and Displ^ Systems. Santa Monica: System Development Corpora- 
tion, December 7, 1966. 

The development of on-line and time -sharing computer usage is described 
as having a significant effect on extending the capability of human performance. A 
study is reported in which problem solvers were provided with computer and dis- 
play aids for aiding short-term memory; eliminating information thought to be 
unimportant; and recoding raw data into other units. Performance for this group 
exceeded a control group, and superiority persisted when the aids were taken away. 
It is concluded from this study and other work discussed that providing people with 
computer and display aids that overcome their limitations - memory and the ability 
to manipulate , recode , and transform data - can expand problem solving and deci- 
sion making abilities tremendously, and has a straightforward application to indus- 
try. 



35. Olson, P. T. Nomograms for Army Manpower Policy Evaluation. Washing- 
ton, D. C.: U;! S. Army Personnel Research Office, Jime 1966. (AD 640 005) 

This report provides a series of charts or nomograms which have been 
developed by the U. S. Army Personnel Research Office for use in evaluating man- 
power policy alternatives. Manpower flow models were developed to represent the 
the Army’s personnel system under varying requirements and restrictions with 
respect to types of duty tour, flow of personnel, duration of tours, allocation of 
personnel, and alternation policy. From models showing the personnel system in 
steady state , a series of nomograms or charts were constructed to facilitate the 
evaluation of alternative rotation policies in the context of contemplated modifica- 
tions in the allocation of forces. The nomograms presented show the steady-state 
relationship between allocation percentages overseas and in the continental U. S. 
for various tour lengths and selected assignment policy alternatives. The charts 
can be used by management personnel to arrive at estimates of the effects of a 
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given policy alternative without recourse to a computer and without working through 
the mathematical developments . The nomograms were employed in a recent staff 
study concerned with the number of men needed in particular service categories 
to support a prescribed rotation policy. 



36 . Proceedings of Air Force/Industry Data Management Symposium. Beverly 
Hills, California, September 28-30, 1965. Norton Air Force Base, 
California: Ballistic Systems Division, 1965. (AD 626 032) 

Several papers presented under the symposium section entitled Advanced 
Concepts for Data Storage, Retrieval, and Communication Techniques cover recent 
technical advances in the storage and retrieval of data with examples of successful 
applications. Subjects discussed were: development of micro-image recording 
techniques; including recent application in videotape, photochromic coated film, 
and thermoplastic film; and use of a computer to provide troubleshooting and fault 
isolation information to electronics mechanics . 



37. Proceedings of the Conference on Engineering Systems For Education and 
Trainmg. June 14-15, 1966. Washington, D. C. : National Security Indils- 
trial Association 1966. 

The views of governmental leaders and prominent researchers on a wide 
variety of educational and manpower issues were presented at this conference. The 
majority of discussion pertained directly to the problems of education, but the 
related issue of performance aiding was touched on by a number of speakers. The 
Honorable Thomas D. Morris, Assistant Secretary of Defense (Manpower), stated 
the objective of providing more information on an "as needed" basis, both for 
training and for use on the job. Colonel Triner of the Air Force stressed the use 
of computer technology for making pre -formulated solutions available to personnel 
in the job situation. 



38. Rees, D. W. & Copeland, N. K. The Effects of Serial Position in Checklist 
Design . Wright -Patterson Air Force Base, Ohio: Wright Air Development 
Center, WADC-TR-59-552, September 1959. (AD 231 990) 

An experiment was conducted to determine (1) whether serial position effects 
occur in the use of checklists, and (2) whether these effects can be modified. Sub- 
jects responded to a serial presentation of instructions by actuating switches. Two 
checklists of instructions were tested: a high and a low generalization list. The 
two criterion measures of performance were time spent observing the list (search 
time) and number of errors made. Results indicated that serial position effect 
does occur when search time is the criterion in checklist utilization. This effect 
can be reduced by isolation techniques; i.e. , typographical arrangement of the 
material . 
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39. Rees, D. W. & Kama, W. N. Size of Tabs: A Factor in Handling of Guides 
and Check -lists . Wright -Pattei-son Air Force Base, Ohio: Wright Air 
Development Center, WADC-TR-59-158, March 1959. (AD 213 595) 

To obtain information pertaining to the design of index tabs as used on check- 
lists to facilitate handling, an experiment was performed in which subjects were 
required to pick up a checklist from a thighboard. The experimental variables 
were (1) size of tabs, (2) encumbrance conditions (bare hands and work clothes, 
gloves, gloves and fully inflated MC-3 partial pressure suits), and (3) tab position 
(top, center, or side of list). Reaction times in using tabs to locate information 
were studied by analysis of variance methods for effect of the variables. Increasing 
tab size resulted in overall improved performance. Presence of tabs was a signif- 
icant improvement and as tab size increased, facility of access increased. 



40. Rigney, J. S., Fromer, R. , Langston, E. T. , & Adams, H. C. Evaluation 
of an Experimental Fault Location Device: I, Fault Location by Radio Op- 
erators . Los Angeles: University of Southern California, Department of 
Psychology, Electronics Personnel Research Group, Technical Report 43, 
August 1965. 

An experimental fault locator (XFL) was developed to be used as a job aid by 
shipboard personnel responsible for corrective maintenance of a radio transceiver. 
The XFL is a small, circular, plastic job-aid which incorporates circuit -front panel 
relationships essential for fault localization. This report describes the XFL and 
presents the results of the first of 2 evaluative studies . Twenty radiomen respon- 
sible for operating the AN/URC-32 aboard their ships were taught how to check out 
the front panel and how to use the XFL in a three -hour training session. Next, 
they were given six test problems to solve. They were allowed as many trials to 
solve each problem as they could complete in one hour. The percentage of subjects 
solving the problems on the first trial was quite high; this percentage increased on 
the second trial, and a very small percentage failed to obtain a correct solution 
within the one hour time limit. These results demonstrate that operators, who have 
had minimum electronics training, can use the XFL successfully to perform fault 
location on the AN/URC-32 transceiver. The XFL opens the way for non-mainte- 
nance personnel to share part of the maintenance burden with the very much over- 
burdened ET, if this is considered to be a desirable management policy. 



41. Rigney, J. W. , Fromer, R., Langston, E. T. & Adams, H. C. Evaluation 
of an Experimental Fault Location Dev ice: II. Fault Location and Isolation 
by Experienced Electronics Technicians . Los Angeles: University of 
Southern California, Department of Psychology, Electronics Personnel 
Research Group, Technical Report 44. September 1965. 

Three groups of ETs were given the same task, that of isolating a malfunction 
to a faulty component. The XFL Group had to use the fault locator (See reference 
No. 40) . The FP Group was given three hours of special instructions on the trans- 
ceiver but no instructions on the fault locator. The Control Group was not given 
any special instructions regarding the transceiver or the fault locator. One-hundred 
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percent of the XFL Group, 90% of the FP Group and 30% of the Control Group 
solved the first problem within the 3 hour time limit. The mean times for suc- 
cessful solution in each group were 51 minutes, 48 minutes, and 130 minutes, 
respectively. Ninety percent of the XFL Group, 50% of the FP Group and 50% of 
the Control Group solved the second problem within the 3 hour time limit. The 
mean times for successful solution in each group were 62 minutes, 66 minutes, 
and 80 minutes, respectively. The results of this study and the previous one 
demonstrate the effectiveness of the fault locator as a maintenance job aid. Both 
Radiomen and Electronics Technicians can be taught to use the device in a very 
short period of time and have shown a high degree of success in its use . 



42. "Robot Instructors Win Friends and Speed Assembly." Factory, November 

1960, 96-99. ’ 

The robot referred to in this article is an Audio-Visual (A-V) Unit. A dis- 
cussion is presented of its development and application to assembly line production 
by several different companies engaged primarily in electronics manufacturing. 
Details on how A-V units operate and their application using varying features of the 
system are portrayed in some length. A key factor in the success of the program 
is in the preparation of slides, clarity of script and the sequencing of the two into 
an instruction which minimizes the need to tax the operator’s memory. At the 
time of this article, one company revealed that with very few exceptions the A-V 
systems have been enthusiastically received. The reason for acceptance appears 
to be that the unit reduces the job to a non-decision making, mechanical process. 
Provided with increased assurance of doing the right job and relieved of the chance 
of misinterpreting text or drawings, the operator is relaxed and experiences less 
fatigue. While the skilled operator appeared to resent the rote process that the 
A-V unit places on him, most other operators indicated a marked appreciation for 
the solitude obtained by eliminating distracting sounds when using earphones. 
Quotations in this article from one company indicated that before A-V use, ex- 
perienced operators achieved only about 60% of work standards with less experi- 
enced operators falling below the 60%. With A-V, practically every operator is 
said to achieve 90 to 100% of work standards consistently with defect rate reduced 
as much as 99% and rework areas virtually eliminated. Another organization 
was quoted as experiencing a 50% cut in learning time; 80% decrease in attention 
required from leadman; improved product quality and increased worker’s versa- 
tility. Still another company was reported to have reduced inspection manpower 
and rework by 85%, and supervision by 70%. 



43. Rogers, J. P. & Thorne, H. W. The Development and Evaluation of An 
Improved Electronics Troubleshooting Manual. Alexandria, Virginia; ’ 
George Washington University, Human Resources Research Office (HumRRO) , 
Technical Report 65-1, March 1965. (AD 614 606) 

A maintenance manual differing in organization and content from conventional 
manuals was developed to test the hypotheses that improvements in those areas 
would directly affect quality and quantity of electronic troubleshooting activities . 

The "test" manual and a "standard" manual were used by two matched groups of 
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military technicians to troubleshoot a set of electronic malfunctions . Results of 
the study indicated that relevancy of content, degree of detail, and functional 
accessibility of information can significantly affect speed and effectiveness of 
electronic troubleshooting activities. 



44. Sheridan, T. B., Duggar, B. C. & Mayer, Sylvia. Design and Use of In- 
formation Systems for Automated On-The-Job TrainingTi. Design of Self- 
Instructional Features. L. G. Hanscom Field. MassT Blfip.trnnin SystPrYig 
Division, U. S. Air Force, ESD-TDR -64-234, January 1964. (AD 602 042) 

This second in a series of five reports is concerned with human engineering 
factors in the design of information systems. Theories, methodology, and design 
principles for implementing self -instructional fea,tures in information systems 
are described. Several techniques and considerations relating to aiding the se- • 
quencing of actions, logic analysis, and query -reasoning behavior are provided. 



45 . Sheridan , T . B . & Mayer , Sylvia . Design and Use of Information Systems 
for Automated On-The-Job Training I. Conceptual and Experimental Ap- 
proaches . L. G. Hanscom Field, ]5Iass: Electronics Systems i)ivision, 

U. S. Air Force, ESD-TDR -64-234, December 1963. (AD 602 041) 

This report discribes the first year’s efforts of a project covering exploratory 
work on automated training for information systems . Exploratory developments 
on laboratory models of automated training subsystems are described. The ulti- 
mate goal is to provide future information systems with the capability to train their 
users on the job . 



46. Shriver, E. L., Fink, C. D., & Trexler, R. C. FORECAST Sy stems AnalyRis 
and Training Methods for Electronics Maintenance Training. Alexandria7Va: ' 
George Washington University, Human Resources Research Office (HumRRO), 
Research Report 13, May 1964. (AD 441 248). 

The report presents results of a continuing study, titled FORECAST, under- 
taken to improve the efficiency of the electronics systems troubleshooting process . 
One of the theses of this study is that there is a discontinuity between trouble- 
shooting theory presented in basic electronics courses and maintenance manuals, 
and the practices that must be followed in on-the-job fault isolation; and that these 
deficiencies in training and job aids can be alleviated by introducing a ’’bridge” 
between theory and practice. The ’’bridge,” consisting of special training courses, 
classroom training aids, and job performance aids was demonstrated to be more 
effective than traditional solutions. Trainees acquired as much information in a 
12 week FORECAST course as those taking a standard 30 week course in one ex- 
perimental study; while graduates of another FORECAST course performed 40% 
better than t: iditional graduates. FORECAST job performance troubleshooting aids 
were developed to permit rapid access of relevant information only. The author 
stated that application of FORECAST training and job aids can significantly affect 
quality of job performance. 
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47. Shriver, E. L. & Trexler, R. C. Application and Test of the FORECAST 

Concept of Electronics Maintenance on Navy LORAN Equipment. Alexandria, 
Va: The George Washington University, Human Resources Research Office 
(HumRRO), Technical Report 65-3, May 1965. (AD 616 753) 

Results of an application study of the FORECAST system to naval electronics 
personnel trained to maintain a LORAN (long range navigational) system are 
presented in this report. The FORECAST concept is a collection of policies, 
strategies, methods, techniques, and services developed for the purpose of im- 
proving the quality of military electronic maintenance. The central theme of the 
FORECAST program is that existing electronics courses and troubleshooting job 
aids presented in maintenance manuals are primarily oriented to electronics 
theory that is of limited value to the real world. Content of training courses and 
job performance aids developed under the FORECAST program ’’bridge” the gap 
between theory and practice. FORECAST training and standard training course 
work were administered to two groups of electronics students. The FORECAST 
group performed significantly better in identifying malfunctions in a performance 
test than those receiving conventional training. 




48. Teel, K. S. & Chaney, F. B. Improving Inspector Performance Through 
Training and Visual Aids . Paper presented to the 74th Annual Convention 
of the American Psychological Association, September 1966. 

The purpose of this study was to evaluate, singly and in combination, the 
effectiveness of a 4-hour training program and a set of visual aids designed to 
improve the performance of machined parts inspectors . The criterion used was 
the percentage of true defects detected in a selected sample of machined parts . 
Findings indicated that: use of training alone resulted in a 32% increase in defects 
detected; use of visual aids alone resulted in a 42% increase; and use of both 
resulted in a 72% increase; while performance of the control group did not change. 



49. Tilley, K. W. , Knight, M. A. & Stainer, F. W. "Training for Fault Diag- 
nosis; The Use of Job Aids," Ergonomics, 9, January 1966, 88. 



This is an abstract of a presentation to the Ergonomics Society Meeting on 
Occupational Training at Berkbeck College, London, on 8 July 1965. Two types of 
job aids, circuit diagrams and fault finding guides, were identified as being under 
study. 



50. Topmiller, D. A. Application of Behavioral Science to Performance Aid 

Development . Wright -Patterson Air Force Base, Onio: Aerospace Medical 
Research Laboratories, Air Force Systems Command, AMRL-TR-65-146, 
August 1965. 

Four classes of variables relevant to behavioral research on the development 
of performance aids are outlined within an historical framework, with supporting 
empirical research. The four classes of variables are: (1) legibility and format 
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variables; (2) variables associated with processing printed numeric information; 

(3) variables associated with troubleshooting informational processing and display 
systems; and (4) variables associated with the physical configurations of performance 
aids. Some predictions are made for future trends in performance -aid behavioral 
studies; e.g. , improving the legibility and format of technical publications, con- 
figuring maintenance information to reduce weight and bulk, conducting cost-effec- 
tiveness studies; and exploiting the rapid data processing functions of computers 
as decision aids. 

51. Travers, R. M. W. (Ed). Research and Theory Related to Audiovisual In- 
formation Transmission. University of Utah, Bureau of Educational Research, 

July 1964. 

A distinction is made between the implicit theory reflected in the products of 
creative designers of audiovisual aids and the principles derived from controlled 
observations and studies of the transmission of information. The findings from the 
empirical work were found to put in question many of the current practices and 
textbook generalizations . It is reported that differences shown for the efficiency of 
audio, visual, and audiovisual media are dependent on such factors as rate of 
presentation, nature of the material, or reading ability. It is also suggested that 
simultaneous transmission through more than one sensory modality presents 
problems of human overloading that may or may not be solvable in any particular 
application. Multiple sensory modality inputs are likely to be of value only when 
the rate of information input is very slow. 



I 52. U. S. Army Test & Evaluation Directorate . Final Report of Test of Audio- 

1 Visual Information System (A -VIS). White Sands Missile Range, New Mexico: 

NIKE X Report No. 65-2, September 1964. (AD 608 308) 

[ Objective and subjective data were collected to evaluate use of an experimental 

I audiovisual information system (A -VIS) for missile system maintenance. Five 

j modes of presentation were used: audiovisual, visual, audio, programmed techni- 

i cal manuals, and conventional technical manuals. The results tended to show a 

I marked superiority of visual and programmed technical manual modes, with 

conventional technical manuals and audiovisual presentations scoring low. The 
principal contribution to the superiority of the automated modes was in information- 
al content. However, the contribution made by the automatic video retrieval 
system was significant, and enhanced the value of the data programs. Several 
hypotheses were advanced to explain the poor showing of audio presentation: (1) 
the use of two senses simultaneously under the requirement to react with some 
time urgency tends to induce a measure of confusion; (2) audio presentation re- 
duces the rate of information transmission and it is not possible to scan audio in 
search of a pertinent detail; (3) the female audio voice used did not inspire con- 
fidence among the technicians . 
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53. Warren, N. D., Schuster, D. H., French, R. S., Latina, R. J. & Nelson, 
R. A. Development and Evaluation of a Troubleshooting Aid for Flight Line 
Maintenance of a Complex Electronic System. Lackland Air Force Base. 
Texas: Air Force Personnel and Training Research Center, AFPTRC-TN- 
58-1. (AD152 110) 

A study was conducted to determine whether an effective and logical trouble- 
shooting aid for complex electronic systems could be developed for use by rela- 
tively inexperienced flightline mechanics. In this study, the investigation of the 
feasibility of such a troubleshooting aid was made for the K Bombing Navigational 
System. A Trouble Locator in booklet format was developed and evaluated. A 
’’procedure -oriented" approach to identify malfunctioning loops supplemented by 
a "symptom -oriented" approach for within -the -loop troubleshooting was found to 
be desirable; both procedures were incorporated in the final Trouble Locator. A 
limited evaluation in the field and on a bench mockup demonstrated that this tj^e 
of troubleshooting aid for complex electronic systems was practical. 



54. Whitehead, L. W. "Simple Nomographs Are Easy to Draw." Power Engi- 
neering, 70, March 1966, p. 90. 

Whitehead suggests that nomographs can be of definite help for repeated 
calculations and to extend results from tests. The construction of a simple nom- 
ograph for adding numbers is described in detail . 



55. Wulff, J. J. & Berry, P. C. "Aids to Job Performance" InR. M. Gagne 
(Ed) Psychological Principles in System Development. New York: Holt 
Rinehart and Winston, 1962, pp. 273-298. 

Wulff and Berry discuss the design and development of job aids for system 
operation. The primary objective of the chapter is to show the relationship between 
the practical problems of job aid design and the more basic psychological principles 
and research problems in human behavior. The chapter provides background in- 
formation on the development and use of job aids in man -machine systems, but the 
major emphasis is upon research problems underlying job aid design. The authors 
draw a parallel between the function that a job aid serves for human performance 
and the function of a program for a computer. Stimulus control of human per- 
formance is conceived as the major psychological problem in the design of job aids. 
Rather than attempting to define the total range of human performances in systems, 
it is argued that job aid research should be directed toward identifying the different 
kinds of performance change that can be effected by stimulus control through job 
aids, and relating these classes of change to the characteristics of job aids. 
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APPENDIX C 

SOURCES OF BACKGROUND INFORMATION 



Organizations and Individuals Active or Interested in the Job Performance Aid 
Field 



Aerospace Medical Research Laboratories 
Wright -Patterson Air Force Base, Ohio 

Dr. R. L. Morgan 

Applied Science Associates 
Valencia, Pennsylvania 

Dr. J. D. Folley 
Dr. T. Elliott 

Autonetics, A Division of North American Aviation, hic. 
Fullerton, California 

Dr. K. S. Teel 
Dr. F. B. Chaney 

U. S. Army Air Defense Human Research Unit 
Human Resources Research Office 
Ft. Bliss, Texas 

Dr. H. L. Ammerman 
Dr. J. P. Rogers 

U. S. Naval Personnel Research Activity 
San Diego, California 

Dr. E. E. Dudek, Technical Director 
Dr. E. I. Jones, Director 
Training Research Laboratory 

University of Southern California 
Los Angeles, California 

Dr. J. W. Rigney, Principal Investigator 
Electronics Personnel Research Group 




C-1 



PHILCO 




WDL Division 



PHILCO-FORD CORPORATION 









I 

i 

I 

I 

I 

I 

■i 



I 

i 



i 







I 

I 



I 

i 

I 













\ 

i 




11 



ij 




WDL-TR3276 



APPENDIX D 

INTERVIEW AND OBSERVATION OUTLINE* 

I. Identifying Data 

A . Organization 

Name 
Address 
Date of Visit 

B. Personnel Contacted 

Name 
Job Title 

II. Description of Performance Aid Used 

A. Description of Task Supported 
Summary - 

Major Task Characteristics 
Procedural 
Repetitive 
Variable 

Frequency per Shift 
Time to Complete 

Major Operations Performed 
Inspection 
Troubleshooting 
Conversion 



i 



I 

1 






■1 

« 

I 



I 

i 




"'I 









* The above form was not intended to cover the specific details of the entire 

interview, but rather was designed to serve as a frame of reference for guiding 
the interview and observation . In keeping with the exploratory nature of the 
survey, the items on the form served as the stimulus for a variety of additional 
questions during the lengthy interview and observation session. 
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Major Operations Performed (Continued) 
Computation 
As sembly/Dis as s embly 
Other 



B. Type of Aid Used 
Narrative 
Checklist 
Tabular 

Calculating Chart 



Graphic Audio 

Drawing Tape 

Film Record 

B&W 
Color 
Slides 
Still Photos 
Motion Silent 
Motion Sound 



C. Characteristics of Aid 
Physical: 
Functional: 
Location: 



D. When and How Used 

E . How Long in Use 

F. Attitudes Toward Performance Aid 

Incmnbents 

Supervisors 



III. Impact on Task Performance 

Quality 

Quantity 

Reliability 

IV. Impact on Job Requirements 

Selection 

Training 
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V. Problems in Using Aid 

Possible Improvement 



VI. Recommendations for New Aids 
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